Next Article in Journal
The Effect of Agriculture Insurance on Agricultural Carbon Emissions in China: The Mediation Role of Low-Carbon Technology Innovation
Next Article in Special Issue
The Impact of COVID-19 on Educational Research: A Bibliometric Analysis
Previous Article in Journal
Opportunities and Potential for Energy Utilization from Agricultural and Livestock Residues in the Region of Thessaly
Previous Article in Special Issue
Sense of Competence and Feelings of Stress of Higher Education Faculty in the Transition to Remote Teaching: What Can We Learn from COVID-19 Pandemic in the Long Run
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Factors Influencing Students’ Continuance Usage Intention with Virtual Classroom during the COVID-19 Pandemic: An Empirical Study

School of Journalism and Communication, Wuhan University, Wuhan 430072, China
*
Author to whom correspondence should be addressed.
Sustainability 2023, 15(5), 4420; https://doi.org/10.3390/su15054420
Submission received: 6 February 2023 / Revised: 24 February 2023 / Accepted: 27 February 2023 / Published: 1 March 2023

Abstract

:
The constant variation of COVID-19 has intensified the spread and recurrence of the epidemic, and education continues to be hard hit in most countries. The virtual classroom has become the main platform to replace the traditional classroom in the COVID-19 pandemic context. Due to the lack of a comprehensive understanding of college students’ perceptions of the platform system, it is essential to explore the factors and mechanisms that influence students’ willingness to use virtual classrooms consistently to improve the learning efficiency and optimize the effect of educational communication during the epidemic. This study integrates the Delone and McLean (D&M) information systems (IS) success model, expectation–confirmation model (ECM), and instructor quality factor to construct an operational model, and it used a structural equation model to analyze the 411 valid samples received from online questionnaires. The results reveal that the determinants of college students’ perceived usefulness of virtual classrooms are service quality, instructor quality, and confirmation, while system quality has no effect on perceived usefulness in the context of the COVID-19 pandemic. Secondly, system quality, service quality, and instructor quality are three critical antecedents of confirmation, and perceived usefulness and confirmation positively affect satisfaction. Finally, perceived usefulness and satisfaction directly affect college students’ continuance intention.

1. Introduction

The global spread of COVID-19 has had a great impact on education in countries around the world [1,2], and governments are rushing to set-up virtual classrooms and collaboration platforms during this period to address the disruption of classes [1,3]. Whereas virtual classrooms provide an alternative way to teach courses during the epidemic, students’ acceptance of the system is key to the success of this online delivery model [4,5]. For example, one study used the technology acceptance model (TAM) to discuss that the ease of use and usefulness of an e-learning system will have a positive impact on students’ willingness to use the system [6]. A scholar verified that performance expectations, effort expectancy, and social influence will positively affect college students’ intention to accept online education services [7]. In addition, the theory of reasoned action (TRA) model [8] and the theory of planned behavior (TPB) model [9] have also been used to explore users’ willingness to use online learning systems.
Although initial acceptance is important to the success of an IS or technology, the long-term viability and ultimate success of an IS depends on its continued use rather than its initial use [10]. The ECM examines the willingness to continue using IS through three dimensions: confirmation, perceived usefulness, and satisfaction. This model has also been widely used in research on measuring users’ continuance intention [11,12,13]. Moreover, it has been suggested that there is a strong relationship between the characteristics of IS and the continuance intention, which include information quality, system quality, and service quality of the D&M IS success model, and these factors play a critical role in users’ use of the system [14]. Meanwhile, instructor quality, as a meaningful indicator to measure teachers’ style and attitude, can have a strong positive impact on users’ perceived usefulness and confirmation, which in turn affects students’ willingness to continue using the learning system [15]. Considering that the virtual classroom discussed in this study is different from a general online learning system, it refers to the dynamic interaction between teachers and learners in a shared online learning space supported by a collaborative learning structure [16]. In the classroom, the teacher mainly provides the teaching content and controls the learning process, and the students, as participants, receive knowledge under the guidance of the teacher. Therefore, we replaced the information quality in the D&M IS success model with the teacher’s quality to more accurately predict the factors that affect students’ continuous use of virtual classrooms.
Given the continued spread of the epidemic in various countries, it is foreseeable that virtual classrooms will be an important way for students to learn about their courses in the future [4]. Therefore, based on existing research results, this study integrated the D&M IS success model and ECM while incorporating instructor quality factor, aiming to explore the important factors that affect Chinese students’ continuous use of virtual classrooms in the context of the continuous variation and mutation of COVID-19.

1.1. Delone and McLean Information Systems Success Model

The IS success model was originally proposed by Delone and McLean [17], and it is mainly used to study the effectiveness or success of IS. The model was initially applied in the study of the use of e-commerce systems and subsequently was gradually used in a variety of ways to measure different types of IS [18]. One study conducted empirical research on e-learning systems, and the results showed that the system quality had a significant impact on both system usage and user satisfaction [19]. In addition, some scholars have combined the IS success model with the unified theory of acceptance and use of technology (UTAUT) model [20,21], task-technology fit (TTF) model [22,23], and TAM model [21,24] to build a new research model to explore users’ acceptance of IS.
Although the IS success model has been verified by studies, scholars still believe that more dimensional factors need to be considered [25,26]. Since a teacher’s attitude is an important indicator for evaluating education-related IS, it can play a role in promoting teaching quality and learning effects [27], and virtual classroom platforms also provide functional support and tools for teacher–student interactions. Therefore, in order to understand the role of teachers in IS, this study incorporated the “instructor quality” factor into the existing model.

1.2. Expectation–Confirmation Model

The expectation–confirmation model (ECM) was developed from expectation–confirmation theory (ECT). In 1980, Oliver [28] proposed the ECT to explore consumer satisfaction and repurchase behavior by constructing the relationship between expectation, perceived performance, confirmation, satisfaction, and repurchase intention. Subsequently, Bhattacherjee applied the model to explain users’ willingness to continue using IS. He suggested that the continuous use behavior of IS users is similar to consumers’ repurchase behavior and proposed the ECM [10].
Since the development of the ECM, it has been validated by scholars and widely used in research on the continuous use of IS in various fields, such as e-government [29], mobile apps [30], digital textbooks [31], learning management systems [32], smart devices [33], and Facebook [34]. In recent years, scholars have applied the ECM to the study of online learning. In a survey of users’ continuous use of e-learning, the researcher explored the predictors of users’ intentions to continue using e-learning, and the empirical results demonstrated that satisfaction had the most significant effect on users’ continuance intention, followed by perceived usefulness, attitude, concentration, subjective norm, and perceived behavior [35]. A study used the ECM as a theoretical basis to explore the factors that influence learners’ continuous use of massive open online courses, the research results show that satisfaction with prior learning experience, confirmation with prior learning experience, and perceived usefulness will all affect learners’ continuance use intention [36]. Existing studies have shown that in the context of the COVID-19 pandemic, the use of the ECM to study college students’ willingness to use virtual classrooms occurs relatively seldom, and both the ECM and the D&M IS success model can be used to explore the factors that affect users’ use of IS. Therefore, this study expanded the two models and constructed a new research model to explore the influencing factors of Chinese college students’ willingness to continue using virtual classrooms.

2. Methodology

2.1. Research Model

Based on the D&M model and the ECM, this study constructed a model of the continuance intention to use virtual classrooms of Chinese college students during the pandemic. Figure 1 represents the study’s framework and hypotheses composed of 7 constructs and 11 hypotheses designed to examine the relationships between these constructs.

2.2. Hypotheses

2.2.1. System Quality, Perceived Usefulness, and Confirmation

System quality represents the quality of the functionality of an IS itself, and it reflects the flexibility, reliability, intuitiveness, sophistication, flexibility, and response times of the system [37,38]. A good system quality can provide useful functions to meet the needs of users to achieve their usage goals; when the system effectively improves users’ learning and working efficiency, they may feel that such a system is useful [39,40]. Moreover, a higher quality system may increase the level of users’ expectations, thus leading to greater confirmation of the IS [15,41]. Accordingly, we can conclude that enhanced system quality in a virtual classroom platform can contribute to a growing degree of students’ perceived usefulness and confirmation. Hence, the following hypotheses were proposed:
Hypothesis 1 (H1).
System quality has a positive impact on college students’ perceived usefulness.
Hypothesis 2 (H2).
System quality has a positive impact on college students’ confirmation.

2.2.2. Service Quality, Perceived Usefulness, and Confirmation

Service quality is another important construct to measure the success of an IS, its measurement includes dimensions such as tangibles, reliability, responsiveness, assurance, and empathy [37,42,43]. When users acquire high-quality services from an IS to solve the problems they encounter in its usage, they may perceive that the system is helpful and tend to continue using it [44,45]. Furthermore, there is a significant relationship between the service quality and the confirmation of users [41]. Studies have shown that the service quality of MOOC platforms has the greatest impact on users’ expectation confirmation [46]. Given the above, a higher service quality of an IS will lead to a higher level of the perceived usefulness and confirmation. Hence, the following hypotheses were proposed:
Hypothesis 3 (H3).
Service quality has a positive impact on college students’ perceived usefulness.
Hypothesis 4 (H4).
Service quality has a positive impact on college students’ confirmation.

2.2.3. Instructor Quality, Perceived Usefulness, and Confirmation

Instructor quality is an indication of a teacher’s teaching style and attitude, which are important factors affecting learners’ behavior [40]. It can be reflected by the instructor’s response timeliness, teaching style, and explanation towards learners through the IS system [47]. A recent study showed that instructor quality is positively associated with learners’ perceived usefulness with an e-learning system [48]. Similar results were also obtained by studies of learning management systems, where instructor involvement can increase the productivity and effectiveness of students’ learning [49]. In addition, researchers have developed a model that takes instructor quality as one of the dimensions affecting users’ confirmation and found that there is a significant relationship between instructor quality and confirmation [15]. Consequently, we may assume that instructors’ timely responses, updated lecture notes, and interactions with students in different aspects are likely to influence students’ perceived usefulness and confirmation with the virtual classroom platforms. Hence, the following hypotheses were proposed:
Hypothesis 5 (H5).
Instructor quality has a positive impact on college students’ perceived usefulness.
Hypothesis 6 (H6).
Instructor quality has a positive impact on college students’ confirmation.

2.2.4. Confirmation, Perceived Usefulness, and Satisfaction

Confirmation refers to the users’ perception of the congruence between their expectation of IS use and its actual performance [10]. Prior studies provide strong support for the relationship between the confirmation of the IS and users’ perceived usefulness [29,32]. Research conducted by Wang, Lin, and Su [13] indicated that the degree of confirmation of students has a large impact on perceived usefulness in using online learning during the pandemic. Likewise, confirmation was also a key determinant of students’ satisfaction with IS systems [50,51,52]. Students will experience a higher level of satisfaction when their expectations are confirmed. Hence, this study assumed that if the students’ experience confirms their expectations of using virtual classrooms, they will feel useful and satisfied. Based on that, the following hypotheses were proposed:
Hypothesis 7 (H7).
Confirmation has a positive impact on college students’ perceived usefulness.
Hypothesis 8 (H8).
Confirmation has a positive impact on college students’ satisfaction.

2.2.5. Perceived Usefulness, Satisfaction, and Continuance Intention

Perceived usefulness is defined as the users’ perceptions of the expected benefits of using an IS [53]. Numerous studies have examined the positive impact of perceived usefulness on satisfaction [10,13,54]. When users feel that they can get help from the IS, their satisfaction level increases. In addition, perceived usefulness is also considered an essential factor that influences individual continuance intention towards using technology [55,56,57]. In the current research context, we conjectured that the higher the perceived usefulness by students using a virtual classroom during the pandemic, the stronger their satisfaction and continuance intention. Consequently, the following hypotheses concerning students’ perceived usefulness were proposed:
Hypothesis 9 (H9).
Perceived usefulness has a positive impact on college students’ satisfaction.
Hypothesis 10 (H10).
Perceived usefulness has a positive impact on college students’ continuance intention.

2.2.6. Satisfaction and Continuance Intention

Satisfaction is an ex post evaluation of users’ initial (trial) experience [10], which is a critical antecedent of individuals’ intentions to reuse a technology or product [34,58,59]. Users with higher levels of satisfaction have stronger intentions to use. Various studies have found that satisfaction with an IS influences students’ continuance intention to use learning technologies in educational contexts [60,61,62]. Based on the aforementioned studies, we can consider that students’ continuance intention of a virtual classroom will increase with their satisfaction. Therefore, the following hypotheses were proposed:
Hypothesis 11 (H11).
Satisfaction has a positive impact on college students’ continuance intention.

2.3. Participants

To explore the factors affecting students’ continuance intention to use virtual classroom during the pandemic, the participants for the present survey research were graduate and undergraduate students from universities in China.

2.4. Research Instrument

To test the research model, the research instrument consisted of two parts. The first part was the students’ demographic information, such as gender, education level, and age. The second part measured the constructs involved in the theoretical model, including system quality, service quality, instructor quality, perceived usefulness, confirmation, satisfaction, and continuance intention based on a 5-point Likert scale (1 = completely disagree, 5 = completely agree). The items of the seven variables were taken from previous studies. The system quality and service quality were adapted from the research work of DeLone and McLean [37], Roca et al. [41], and Hassanzadeh et al. [43]. The instructor quality was informed by Mtebe and Raphael [27]. The perceived usefulness, confirmation, satisfaction, and continuance intention were measured through a scale developed by Bhattacherjee [10], Thong et al. [29], Lee and Kwon [55], and Mouakket [34]. Table A1 presents the questionnaire constructs and related items.

2.5. Data Collection

The questionnaire was distributed via the professional online questionnaire survey platform Wenjuanxing (www.wjx.cn, accessed on 31 December 2022). We sent the questionnaire to Chinese university students through social media. The quality of the questionnaire was strictly controlled. Finally, we collected a total of 411 questionnaires after excluding incomplete or invalid answers. Table 1 shows the sociodemographic characteristics of the participants. Of all these participants, males accounted for 28.22% (n = 116) and females 71.78% (n = 295); undergraduate students were 70.32% (n = 289), and graduate students were 29.68% (n = 122); and the participants aged 18–24 reached 73.24% (n = 301), and those aged 25–29 were 26.76% (n = 110).

3. Data Analysis and Results

This study applied a partial least squares structural equation modeling (PLS-SEM) statistical approach using SmartPLS 3.2.9 software (SmartPLS GmbH, Oststeinbek, Germany) to assess and validate the research model. Compared with the covariance-based SEM method, PLS-SEM has a powerful exploration capacity with complex models and small sample sizes [63,64]. The data analysis followed the two-step assessment procedure, including a measurement model and structural model [64,65].

3.1. Analysis of the Measurement Model

The construct reliability and validity were used to evaluate the measurement model [64], and the main indicators for the assessment were the internal consistency, convergent validity, and discriminant validity. The internal consistency is usually measured using Cronbach’s α and composite reliability. As shown in Table 2, the composite reliability for all of the latent variables ranged from 0.849 to 0.947, and all Cronbach’s α were above the threshold of 0.7, which sufficiently indicates good internal consistency and a satisfactory level of reliability.
The convergent validity was tested by the standardized factor loadings and average variance extracted (AVE) [66]. As shown in Table 2, the standardized factor loading of each item in this study exceeded the recommended value of 0.5, and all AVE values were higher than the suggested threshold of 0.5. This means all constructs had fairly good convergent validity. In addition, discriminant validity is the extent a latent variable discriminates from other latent variables, with the assessment through the square root of the average variance extracted (AVE). According to Hair et al. [64], when comparing the square roots of the AVEs with the other values on each column, the square roots of the AVEs for each latent variable must be greater than any correlation relating to each latent variable. Table 3 presents the discriminant validity for all seven constructs, which confirmed the qualified requirement for assessing the discriminant validity and could be useful for analyzing the structural model in the next section.

3.2. Analysis of the Structural Model

The structural model was examined using the bootstrapping algorithm with the resample 5000 times to discover whether it confirmed the model path and hypothesis [67]. The outcome of the bootstrapping methods is documented in Table 4.
Based on our investigation, we found that all proposed hypotheses were accepted, with the exception of H1. To be more specific, system quality was found to exert a positive impact on confirmation (β = 0.261, p < 0.001), supporting H2. Service quality was significantly associated with perceived usefulness (β = 0.219, p < 0.001) and confirmation (β = 0.249, p < 0.001), supporting H3 and H4. Instructor quality was verified positively with perceived usefulness (β = 0.388, p < 0.001) and confirmation (β = 0.245, p < 0.001), which supports H5 and H6. Confirmation significantly influenced perceived usefulness (β = 0.312, p < 0.001) and satisfaction (β = 0.300, p < 0.001), supporting H7 and H8. Perceived usefulness was found to exert a positive impact on satisfaction (β = 0.568, p < 0.001) and continuance intention (β = 0.411, p < 0.001), which supports H9 and H10. Satisfaction was positively significant for continuance intention (β = 0.428, p < 0.001), thus supporting H11. However, the relationship between system quality and perceived usefulness (β = 0.004, p = 0.945) was found to be statistically insignificant, and H1 is not supported.
The R2 was used to represent the amount of explained variance of the endogenous constructs. As it is shown in Figure 2, the R2 value for confirmation was 0.451, for perceived usefulness it was 0.632, for satisfaction it was 0.693, and for continuance intention it was 0.631. The results indicate that the research model proposed for this study achieved an acceptable level of explanatory power and provides valuable statistical results.

4. Discussion

This research aimed to explore the factors affecting the continuous use of virtual classrooms by college students in China during the COVID-19 pandemic. The research model in this study was developed based on the D&M IS success model and the ECM, and it was used to test 411 college students from different China universities.
The empirical results demonstrate that system quality has no relationship with perceived usefulness but has a positive effect on confirmation. Hypothesis H1 is rejected, and H2 is accepted. On the one hand, this finding shows that the quality of the virtual classroom system does not bring significant help to college students in its use. A rationalization is that the sudden online migration slows down the internet connection, and problems with the system in terms of stability, security, and compatibility have an impact on the user experience [68]. Similar findings have been obtained in recent years [69,70]. On the other hand, system quality is an important determinant of confirmation, and this result shows that providing students with a high-quality system could potentially increase their level of confirmation in virtual classrooms. Therefore, we can expect that the system qualities, such as flexibility, reliability, and interactivity, can effectively meet the needs of college students and help improve their expectations for virtual classrooms, which is consistent with previous research findings [71]. In summary, whereas the responsiveness of the system affects students’ perceptions of its usefulness, other features of the system can help them confirm expectations through the use of virtual classrooms.
Service quality is a prominent predictor of perceived usefulness and confirmation (H3 and H4). The results indicate that the higher the quality of service, the higher the usefulness and confirmation of using the virtual classroom. For example, in terms of service quality, such as providing operational guidance on how to use the platform, providing relevant services according to a specific context, and being able to solve technical problems during use so that students can effectively complete classroom tasks, the usefulness and value of the system will be recognized [40], allowing students to have their expectations of the platform validated [41].
Similarly, hypotheses H5 and H6 are also accepted. This confirms that instructor quality is an essential factor in perceived usefulness and confirmation. These findings indicated that timely feedback and updated lecture notes from teachers using virtual classrooms are important in helping college students acquire knowledge of the course content more effectively [48,72]. Furthermore, students are expected to develop a high level of confirmation of the virtual classrooms when teachers respond to learners’ questions and concerns [15]. These results further emphasize that teachers are an important success factor for students in online learning environments [48].
H7 and H8 are supported, and confirming is a significant contributing factors of users’ perceived usefulness and satisfaction. This result corresponds to the findings of previous studies [10,52,54]. This implies that, as students gain more experience and familiarity with the virtual classroom, their belief in the usefulness and satisfaction of the system will increase if they feel that the functions and services provided by the system can improve their learning performance and efficiency, helping them achieve learning goals smoothly and successfully with less effort [12].
In addition, our results show that perceived usefulness has a positive effect on satisfaction, supporting H9. Although both perceived usefulness and confirmation are predictors of satisfaction, perceived usefulness has a stronger effect on satisfaction (β = 0.568) than confirmation (β = 0.300). This result can be attributed to the fact that college students focus on the value of the usefulness provided by the virtual classroom that can help them successfully acquire classroom knowledge and less on whether the system meets their expectations. Similar results were obtained by Cheng [15] in research on online learning.
Finally, the study finds that students’ willingness to continue using the virtual classroom is influenced by the perceived usefulness and satisfaction, thus confirming H10 and H11. These results are consistent with other researchers investigating similar relationships [31,34,56]. In other words, when students have a higher perception of the usefulness of the virtual classroom, their willingness to continue using the system will correspondingly be higher. Likewise, students who are satisfied with using the virtual classroom are more likely to continue using it in the future.

5. Implications

5.1. Theoretical Implications

This study developed an integrated model to better understand college students’ intentions to continue using virtual classrooms from widely accepted theories (D&M IS success model and ECM). This attempt can provide theoretical contributions to the existing research field. In this research, we considered system quality, service quality, and instructor quality as antecedent factors affecting perceived usefulness and confirmation, which are of value in explaining the continued use of virtual classrooms, especially in the context of the continuous variation and mutation of COVID-19. In addition, this study sought to enrich the empirical research on the impact of the COVID-19 pandemic on education and plays a role in complementing and improving the theoretical system of the intention to continue using virtual classrooms in terms of the potential risks.

5.2. Practical Implications

The findings of this study have several practical implications for virtual classroom developers to refine the system functionality and for teachers to improve their teaching style. First, from the perspective of virtual classroom developers, it is necessary to pay attention to the impact of system quality and service quality on user confirmation. Therefore, it is recommended that developers enhance the system maintenance, improve the operating speed, strive to develop system functions, optimize the operating procedures, and provide more intuitive and easy-to-use services to meet the initial expectations of college students regarding the platform. Moreover, system quality is also an essential factor affecting perceived usefulness, which means that developers need to ensure that the system can run smoothly during use and avoid audio or display lag so that users can feel the usefulness of the virtual classroom in improving the classroom efficiency.
Second, from the perspective of the teacher, in addition to strengthening the interactions with students in the classroom, it is recommended to establish a corresponding teacher evaluation mechanism, which can include the evaluation of the teacher’s style and provide a channel for students to express their opinions so that the teacher can make timely adjustments in response to the needs, which would enable students to feel the actual usefulness of the system and meet their expectations.

6. Limitations and Future Directions

The limitations of the current study can be described as follows. First, the respondents of this study were mainly college students. Future research can also consider primary school students, junior high school students, and high school students as research objects, which may produce different results among different groups. Secondly, the current study focused on participants in China, which may not suit the generalizability of the results. To draw more general conclusions, future studies may conduct surveys in different countries or regions under various situations and perform multiple-group analyses to check for possible cultural and technical differences. Third, the study mainly used quantitative research, and qualitative research could be applied in the future to investigate the issues in more depth.

7. Conclusions

This study provides insight into the factors that affect the willingness of Chinese college students to continue to use virtual classrooms in the context of the COVID-19 epidemic. It proposed a new research model based on the D&M IS success model, ECM, and instructor quality. The results show that system quality, service quality, and instructor quality have a significant effect on the confirmation, and service quality and instructor quality have a positive impact on perceived usefulness. Perceived usefulness and confirmation explain college students’ satisfaction with the use of the virtual classroom system, which in turn leads to college students’ continuous use intention. This study combined both internal factors (i.e., system quality and service quality) and external factors (i.e., teacher quality) to reveal that what influenced college students’ continuous use of the virtual classroom during COVID-19 not only derived from the characteristics of the system itself but also from the instructional style presented by the teacher in virtual classrooms. This finding is of great significance for system developers and teachers to improve and optimize the existing situation to create a good virtual classroom environment for college students during COVID-19 in order to increase their continuance intention.

Author Contributions

Conceptualization, X.H.; Data curation, X.H.; Formal analysis, X.H.; Investigation, H.Z.; Methodology, X.H. and H.Z.; Software, X.H.; Validation, X.H. and H.Z.; Writing—original draft, X.H.; Writing—review and editing, X.H. and H.Z.; Supervision, H.Z. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

This study was conducted according to the guidelines of the Declaration of Helsinki and Measures for the Ethical Review of Biomedical Research Involving Humans released by the National Health and Family Planning Commission (China), and it was approved by the Academic Committee of the School of Journalism and Communication, Wuhan University.

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

Data are available upon special request from the corresponding author.

Conflicts of Interest

The authors declare no conflict of interest.

Appendix

Table A1. Measurement items.
Table A1. Measurement items.
ConstructMeasurement Items
System QualityThe virtual classroom responds quickly and runs smoothly in general.
The virtual classroom can be used stably in mobile terminals and computer terminals.
The virtual classroom provides different learning styles (e.g., flash animation, video, audio, text, and simulation), and they are interesting and appropriate in my study.
The virtual classroom realizes interactions between teachers and students in the classroom.
Service QualityThere are enough and clear instructions/training on how to use the virtual classroom.
The virtual classroom has a good interface to communicate my needs.
The virtual classroom provides technical support and online assistance.
The virtual classroom provides the right solution to my request.
Instructor QualityThe instructor responds timely to learners via the virtual classroom platform.
The instructor updates lecture notes on the virtual classroom platform frequently.
The instructor responds to learners’ questions via the virtual classroom platform promptly.
The instructor responds to learners’ concerns via the virtual classroom platform in a friendly manner.
Perceived UsefulnessUsing the virtual classroom improves my performance in the classroom.
Using the virtual classroom improves my efficiency in study.
The virtual classroom helps me communicate with teachers more conveniently.
Overall, I find virtual classroom to be useful for study.
ConfirmationMy experience with using the virtual classroom was better than what I expected.
The service level provided by the virtual classroom was better than what I expected.
Overall, most of my expectations from using the virtual classroom were confirmed.
SatisfactionMy overall experience of the virtual classroom use was very satisfying.
My overall experience of the virtual classroom use was very pleasant.
Overall, I am satisfied with the quality of the virtual classroom system.
Continuance IntentionI intend to continue using the virtual classroom rather than discontinue its use.
I intend to continue using the virtual classroom for my study in future.
My intention is to continue using the virtual classroom rather than use any alternative means.

References

  1. Reuge, N.; Jenkins, R.; Brossard, M.; Soobrayan, B.; Mizunoya, S.; Ackers, J.; Jones, L.; Taulo, W.G. Education response to COVID 19 pandemic, a special issue proposed by UNICEF: Editorial review. Int. J. Educ. Dev. 2021, 87, 102485. [Google Scholar] [CrossRef]
  2. Kuhfeld, M.; Soland, J.; Tarasawa, B.; Johnson, A.; Ruzek, E.; Liu, J. Projecting the Potential Impact of COVID-19 School Closures on Academic Achievement. Educ. Res. 2020, 49, 549–565. [Google Scholar] [CrossRef]
  3. Favale, T.; Soro, F.; Trevisan, M.; Drago, I.; Mellia, M. Campus traffic and e-Learning during COVID-19 pandemic. Comput. Netw. 2020, 176, 107290. [Google Scholar] [CrossRef]
  4. Neuwirth, L.S.; Jović, S.; Mukherji, B.R. Reimagining higher education during and post-COVID-19: Challenges and opportunities. J. Adult Contin. Educ. 2020, 27, 141–156. [Google Scholar] [CrossRef]
  5. Wang, R.; Han, J.; Liu, C.; Xu, H. How Do University Students’ Perceptions of the Instructor’s Role Influence Their Learning Outcomes and Satisfaction in Cloud-Based Virtual Classrooms During the COVID-19 Pandemic? Front. Psychol. 2021, 12, 627443. [Google Scholar] [CrossRef]
  6. Baber, H. Modelling the acceptance of e-learning during the pandemic of COVID-19-A study of South Korea. Int. J. Manag. Educ. 2021, 19, 100503. [Google Scholar] [CrossRef]
  7. PARK, M.-J.; LEE, J.-K. Investigation of college students’ intention to accept online education services: An application of the UTAUT model in Korea. J. Asian Financ. Econ. Bus. 2021, 8, 327–336. [Google Scholar] [CrossRef]
  8. Cao, J.; Yang, T.; Lai, I.K.-W.; Wu, J. Student acceptance of intelligent tutoring systems during COVID-19: The effect of political influence. Int. J. Elec. Eng. Educ. 2021. [CrossRef]
  9. Kim, E.-J.; Kim, J.J.; Han, S.-H. Understanding Student Acceptance of Online Learning Systems in Higher Education: Application of Social Psychology Theories with Consideration of User Innovativeness. Sustainability 2021, 13, 896. [Google Scholar] [CrossRef]
  10. Bhattacherjee, A. Understanding Information Systems Continuance: An Expectation-Confirmation Model. MIS Q. 2001, 25, 351–370. [Google Scholar] [CrossRef]
  11. Ibrahim, Y.; Hidayat-ur-Rehman, I. COVID-19 crisis and the continuous use of virtual classes. Int. J. Adv. Appl. Sci. 2021, 8, 117–129. [Google Scholar] [CrossRef]
  12. Chauhan, S.; Goyal, S.; Bhardwaj, A.K.; Sergi, B.S. Examining continuance intention in business schools with digital classroom methods during COVID-19: A comparative study of India and Italy. Behav. Inf. Technol. 2022, 41, 1596–1619. [Google Scholar] [CrossRef]
  13. Wang, T.; Lin, C.-L.; Su, Y.-S. Continuance Intention of University Students and Online Learning during the COVID-19 Pandemic: A Modified Expectation Confirmation Model Perspective. Sustainability 2021, 13, 4586. [Google Scholar] [CrossRef]
  14. Pang, S.; Bao, P.; Hao, W.; Kim, J.; Gu, W. Knowledge Sharing Platforms: An Empirical Study of the Factors Affecting Continued Use Intention. Sustainability 2020, 12, 2341. [Google Scholar] [CrossRef] [Green Version]
  15. Cheng, Y.-M. Extending the expectation-confirmation model with quality and flow to explore nurses’ continued blended e-learning intention. Inf. Technol. People 2014, 27, 230–258. [Google Scholar] [CrossRef]
  16. Arbaugh, J.B. Virtual Classroom Characteristics and Student Satisfaction with Internet-Based MBA Courses. J. Manag. Educ. 2000, 24, 32–54. [Google Scholar] [CrossRef]
  17. DeLone, W.H.; McLean, E.R. Information systems success: The quest for the dependent variable. Inf. Syst. Res. 1992, 3, 60–95. [Google Scholar] [CrossRef] [Green Version]
  18. Yakubu, N.; Dasuki, S. Assessing eLearning systems success in Nigeria: An application of the DeLone and McLean information systems success model. J. Inf. Technol. Educ.-Res. 2018, 17, 183–203. [Google Scholar] [CrossRef] [Green Version]
  19. Efiloğlu Kurt, Ö. Examining an e-learning system through the lens of the information systems success model: Empirical evidence from Italy. Educ. Inf. Technol. 2019, 24, 1173–1184. [Google Scholar] [CrossRef]
  20. Lee, Y.-P.; Tsai, H.-Y.; Ruangkanjanases, A. The Determinants for Food Safety Push Notifications on Continuance Intention in an E-Appointment System for Public Health Medical Services: The Perspectives of UTAUT and Information System Quality. Int. J. Environ. Res. Public Health 2020, 17, 8287. [Google Scholar] [CrossRef] [PubMed]
  21. Bessadok, A. Analyzing student aspirations factors affecting e-learning system success using a structural equation model. Educ. Inf. Technol. 2022, 27, 9205–9230. [Google Scholar] [CrossRef] [PubMed]
  22. Tam, C.; Oliveira, T. Understanding the impact of m-banking on individual performance: DeLone & McLean and TTF perspective. Comput. Hum. Behav. 2016, 61, 233–244. [Google Scholar] [CrossRef]
  23. Isaac, O.; Aldholay, A.; Abdullah, Z.; Ramayah, T. Online learning usage within Yemeni higher education: The role of compatibility and task-technology fit as mediating variables in the IS success model. Comput. Educ. 2019, 136, 113–129. [Google Scholar] [CrossRef]
  24. Mohammadi, H. Investigating users’ perspectives on e-learning: An integration of TAM and IS success model. Comput. Hum. Behav. 2015, 45, 359–374. [Google Scholar] [CrossRef]
  25. Wang, Y.-S.; Wang, H.-Y.; Shee, D.Y. Measuring e-learning systems success in an organizational context: Scale development and validation. Comput. Hum. Behav. 2007, 23, 1792–1808. [Google Scholar] [CrossRef]
  26. Aldholay, A.; Isaac, O.; Abdullah, Z.; Abdulsalam, R.; Al-Shibami, A.H. An extension of Delone and McLean IS success model with self-efficacy. Int. J. Inf. Learn. Technol. 2018, 35, 285–304. [Google Scholar] [CrossRef]
  27. Mtebe, J.S.; Raphael, C. Key factors in learners’ satisfaction with the e-learning system at the University of Dar es Salaam, Tanzania. Australas. J. Educ. Technol. 2018, 34, 34. [Google Scholar] [CrossRef] [Green Version]
  28. Oliver, R.L. A Cognitive Model of the Antecedents and Consequences of Satisfaction Decisions. J. Mark. Res. 1980, 17, 460–469. [Google Scholar] [CrossRef]
  29. Thong, J.Y.L.; Hong, S.-J.; Tam, K.Y. The effects of post-adoption beliefs on the expectation-confirmation model for information technology continuance. Int. J. Hum.-Comput. Stud. 2006, 64, 799–810. [Google Scholar] [CrossRef]
  30. Tam, C.; Santos, D.; Oliveira, T. Exploring the influential factors of continuance intention to use mobile Apps: Extending the expectation confirmation model. Inf. Syst. Front. 2020, 22, 243–257. [Google Scholar] [CrossRef]
  31. Joo, Y.J.; Park, S.; Shin, E.K. Students’ expectation, satisfaction, and continuance intention to use digital textbooks. Comput. Hum. Behav. 2017, 69, 83–90. [Google Scholar] [CrossRef]
  32. Ashrafi, A.; Zareravasan, A.; Rabiee Savoji, S.; Amani, M. Exploring factors influencing students’ continuance intention to use the learning management system (LMS): A multi-perspective framework. Interact. Learn. Environ. 2022, 30, 1475–1497. [Google Scholar] [CrossRef]
  33. Park, E. User acceptance of smart wearable devices: An expectation-confirmation model approach. Telemat. Inform. 2020, 47, 101318. [Google Scholar] [CrossRef]
  34. Mouakket, S. Factors influencing continuance intention to use social network sites: The Facebook case. Comput. Hum. Behav. 2015, 53, 102–110. [Google Scholar] [CrossRef]
  35. Lee, M.-C. Explaining and predicting users’ continuance intention toward e-learning: An extension of the expectation–confirmation model. Comput. Educ. 2010, 54, 506–516. [Google Scholar] [CrossRef]
  36. Zhou, J. Exploring the factors affecting learners’ continuance intention of MOOCs for online collaborative learning: An extended ECM perspective. Australas. J. Educ. Technol. 2017, 33, 123–135. [Google Scholar] [CrossRef] [Green Version]
  37. William, H.D.; Ephraim, R.M. The DeLone and McLean Model of Information Systems Success: A Ten-Year Update. J. Manag. Inform. Syst. 2003, 19, 9–30. [Google Scholar] [CrossRef]
  38. Petter, S.; DeLone, W.; McLean, E. Measuring information systems success: Models, dimensions, measures, and interrelationships. Eur. J. Inform. Syst. 2008, 17, 236–263. [Google Scholar] [CrossRef]
  39. Pituch, K.A.; Lee, Y.-K. The influence of system characteristics on e-learning use. Comput. Educ. 2006, 47, 222–244. [Google Scholar] [CrossRef]
  40. Cheng, Y.M. Effects of quality antecedents on e-learning acceptance. Internet Res. 2012, 22, 361–390. [Google Scholar] [CrossRef] [Green Version]
  41. Roca, J.C.; Chiu, C.-M.; Martínez, F.J. Understanding e-learning continuance intention: An extension of the Technology Acceptance Model. Int. J. Hum.-Comput. Stud. 2006, 64, 683–696. [Google Scholar] [CrossRef] [Green Version]
  42. Petter, S.; McLean, E.R. A meta-analytic assessment of the DeLone and McLean IS success model: An examination of IS success at the individual level. Inf. Manag. 2009, 46, 159–166. [Google Scholar] [CrossRef]
  43. Hassanzadeh, A.; Kanaani, F.; Elahi, S. A model for measuring e-learning systems success in universities. Expert Syst. Appl. 2012, 39, 10959–10966. [Google Scholar] [CrossRef]
  44. Saeed, K.A.; Abdinnour-Helm, S. Examining the effects of information system characteristics and perceived usefulness on post adoption usage of information systems. Inf. Manag. 2008, 45, 376–386. [Google Scholar] [CrossRef]
  45. Chen, C.-Y.; Shih, B.-Y.; Yu, S.-H. Disaster prevention and reduction for exploring teachers’ technology acceptance using a virtual reality system and partial least squares techniques. Nat. Hazards 2012, 62, 1217–1231. [Google Scholar] [CrossRef]
  46. Gu, W.; Xu, Y.; Sun, Z.-J. Does MOOC Quality Affect Users’ Continuance Intention? Based on an Integrated Model. Sustainability 2021, 13, 12536. [Google Scholar] [CrossRef]
  47. Choi, D.H.; Kim, J.; Kim, S.H. ERP training with a web-based electronic learning system: The flow theory perspective. Int. J. Hum. Comput. Stud. 2007, 65, 223–243. [Google Scholar] [CrossRef]
  48. Al-Fraihat, D.; Joy, M.; Masa’deh, R.e.; Sinclair, J. Evaluating E-learning systems success: An empirical study. Comput. Hum. Behav. 2020, 102, 67–86. [Google Scholar] [CrossRef]
  49. Klobas, J.E.; McGill, T.J. The role of involvement in learning management system success. J. Comput. High. Educ. 2010, 22, 114–134. [Google Scholar] [CrossRef]
  50. Limayem, M.; Cheung, C.M.K. Understanding information systems continuance: The case of Internet-based learning technologies. Inf. Manag. 2008, 45, 227–232. [Google Scholar] [CrossRef]
  51. Lin, T.-C.; Wu, S.; Hsu, J.S.-C.; Chou, Y.-C. The integration of value-based adoption and expectation–confirmation models: An example of IPTV continuance intention. Decis. Support Syst. 2012, 54, 63–75. [Google Scholar] [CrossRef]
  52. Stone, R.W.; Baker-Eveleth, L. Students’ expectation, confirmation, and continuance intention to use electronic textbooks. Comput. Hum. Behav. 2013, 29, 984–990. [Google Scholar] [CrossRef]
  53. Davis, F.D. Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology. MIS Q. 1989, 13, 319–340. [Google Scholar] [CrossRef] [Green Version]
  54. Joo, S.; Choi, N. Understanding users’ continuance intention to use online library resources based on an extended expectation-confirmation model. Electron. Libr. 2016, 34, 554–571. [Google Scholar] [CrossRef]
  55. Lee, Y.; Kwon, O. Intimacy, familiarity and continuance intention: An extended expectation–confirmation model in web-based services. Electron. Commer. Res. Appl. 2011, 10, 342–357. [Google Scholar] [CrossRef]
  56. Tang, J.-t.E.; Tang, T.-I.; Chiang, C.-H. Blog learning: Effects of users’ usefulness and efficiency towards continuance intention. Behav. Inf. Technol. 2014, 33, 36–50. [Google Scholar] [CrossRef]
  57. Nascimento, B.; Oliveira, T.; Tam, C. Wearable technology: What explains continuance intention in smartwatches? J. Retail. Consum. Serv. 2018, 43, 157–169. [Google Scholar] [CrossRef]
  58. Hong, S.; Thong, J.Y.L.; Tam, K.Y. Understanding continued information technology usage behavior: A comparison of three models in the context of mobile internet. Decis. Support Syst. 2006, 42, 1819–1834. [Google Scholar] [CrossRef]
  59. Choi, G.; Nam, C.; Kim, S. The impacts of technology platform openness on application developers’ intention to continuously use a platform: From an ecosystem perspective. Telecommun. Policy 2019, 43, 140–153. [Google Scholar] [CrossRef]
  60. Ifinedo, P. Investigating the Antecedents of Continuance Intention of Course Management Systems Use among Estonian Undergraduates. Int. J. Inf. Commun. Technol. Educ. 2007, 3, 76–92. [Google Scholar] [CrossRef]
  61. Liao, Y.-W.; Huang, Y.-M.; Wang, Y.-S. Factors Affecting Students’ Continued Usage Intention Toward Business Simulation Games: An Empirical Study. J. Educ. Comput. Res. 2015, 53, 260–283. [Google Scholar] [CrossRef]
  62. Ramadhan, A.; Hidayanto, A.N.; Salsabila, G.A.; Wulandari, I.; Jaury, J.A.; Anjani, N.N. The effect of usability on the intention to use the e-learning system in a sustainable way: A case study at Universitas Indonesia. Educ. Inf. Technol. 2022, 27, 1489–1522. [Google Scholar] [CrossRef]
  63. Henseler, J.; Ringle, C.M.; Sinkovics, R.R. The use of partial least squares path modeling in international marketing. In New Challenges to International Marketing; Sinkovics, R.R., Ghauri, P.N., Eds.; Advances in International Marketing; Emerald Group Publishing Limited: Bingley, UK, 2009; Volume 20, pp. 277–319. [Google Scholar]
  64. Hair, J.F.; Hult, G.T.M.; Ringle, C.; Sarstedt, M. A Primer on Partial Least Squares Structural Equation Modeling (PLS-SEM), 2nd ed.; SAGE Publications, Inc.: Thousand Oaks, CA, USA, 2017; ISBN 9781483377445. [Google Scholar]
  65. Schumacker, R.E.; Lomax, R.G. A Beginner’s Guide to Structural Equation Modeling; Psychology Press: London, UK, 2004. [Google Scholar]
  66. Gefen, D.; Straub, D.; Boudreau, M.-C. Structural equation modeling and regression: Guidelines for research practice. Commun. Assoc. Inf. Syst. 2000, 4, 7. [Google Scholar] [CrossRef] [Green Version]
  67. Hair, J.; Hollingsworth, C.L.; Randolph, A.B.; Chong, A.Y.L. An updated and expanded assessment of PLS-SEM in information systems research. Ind. Manag. Data Syst. 2017, 117, 442–458. [Google Scholar] [CrossRef]
  68. Chen, T.; Peng, L.; Jing, B.; Wu, C.; Yang, J.; Cong, G. The Impact of the COVID-19 Pandemic on User Experience with Online Education Platforms in China. Sustainability 2020, 12, 7329. [Google Scholar] [CrossRef]
  69. Chen, R.-F.; Hsiao, J.-L. An Empirical Study of Physicians’ Acceptance of Hospital Information Systems in Taiwan. Telemed. e-Health 2012, 18, 120–125. [Google Scholar] [CrossRef]
  70. Gupta, P.; Prashar, S.; Vijay, T.S.; Parsad, C. Examining the influence of antecedents of continuous intention to use an informational app: The role of perceived usefulness and perceived ease of use. Int. J. Bus Inf. Syst. 2021, 36, 270–287. [Google Scholar] [CrossRef]
  71. Dağhan, G.; Akkoyunlu, B. Modeling the continuance usage intention of online learning environments. Comput. Hum. Behav. 2016, 60, 198–211. [Google Scholar] [CrossRef]
  72. Lee, B.-C.; Yoon, J.-O.; Lee, I. Learners’ acceptance of e-learning in South Korea: Theories and results. Comput. Educ. 2009, 53, 1320–1329. [Google Scholar] [CrossRef]
Figure 1. Research model.
Figure 1. Research model.
Sustainability 15 04420 g001
Figure 2. Results of the research model.
Figure 2. Results of the research model.
Sustainability 15 04420 g002
Table 1. Sociodemographic characteristics of the participants (n = 411).
Table 1. Sociodemographic characteristics of the participants (n = 411).
ItemContentFrequencyPercentage
GenderMale11628.22%
Female29571.78%
EducationUndergraduate student28970.32%
Graduate student12229.68%
Age18–2430173.24%
25–2911026.76%
Table 2. Factor loading, Cronbach’s α, composite reliability, and AVE.
Table 2. Factor loading, Cronbach’s α, composite reliability, and AVE.
ConstructItemFactor LoadingCronbach’s αComposite ReliabilityAVE
System Quality (SYQ)SYQ1
SYQ2
SYQ3
SYQ4
0.771
0.771
0.759
0.758
0.7640.8490.584
Service Quality (SEQ)SEQ1
SEQ2
SEQ3
SEQ4
0.847
0.847
0.897
0.863
0.8860.9220.746
Instructor Quality (INQ)INQ1
INQ2
INQ3
INQ4
0.887
0.884
0.849
0.866
0.8950.9270.760
Perceived Usefulness (PU)PU1
PU2
PU3
PU4
0.853
0.884
0.904
0.721
0.8620.9070.712
Confirmation (CON)CON1
CON2
CON3
0.828
0.893
0.893
0.8420.9050.760
Satisfaction (SAT)SAT1
SAT2
SAT3
0.906
0.947
0.924
0.9170.9470.857
Continuance Intention (CI)CI1
CI2
CI3
0.892
0.852
0.872
0.8430.9050.761
Table 3. Results of the discriminant validity (Fornell–Larcker criterion).
Table 3. Results of the discriminant validity (Fornell–Larcker criterion).
CICONINQPUSATSEQSYQ
CI0.872
CON0.5690.872
INQ0.6330.5960.872
PU0.7500.6740.7150.844
SAT0.7540.7190.7040.7940.926
SEQ0.6030.5820.6280.6740.6100.864
SYQ0.5930.6150.7450.6360.6790.6870.764
Table 4. Structural path analysis result.
Table 4. Structural path analysis result.
HypothesisRelationshipOriginal Sample (O)Sample Mean (M)Standard Deviation (STDEV)T Statistics (|O/STDEV|)p-ValuesResult
H1SYQ→PU0.0040.0010.0650.0690.945Not supported
H2SYQ→CON0.2610.2630.0703.8740.000Supported
H3SEQ→PU0.2190.2200.0444.9030.000Supported
H4SEQ→CON0.2490.2500.0594.2680.000Supported
H5INQ→PU0.3880.3890.0616.2580.000Supported
H6INQ→CON0.2450.2420.0723.5780.000Supported
H7CON→PU0.3120.3150.0555.8880.000Supported
H8CON→SAT0.3000.3380.0535.4590.000Supported
H9PU→SAT0.5680.5670.04910.2470.000Supported
H10PU→CI0.4110.4090.0556.9700.000Supported
H11SAT→CI0.4280.4280.0606.5840.000Supported
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Huang, X.; Zhi, H. Factors Influencing Students’ Continuance Usage Intention with Virtual Classroom during the COVID-19 Pandemic: An Empirical Study. Sustainability 2023, 15, 4420. https://doi.org/10.3390/su15054420

AMA Style

Huang X, Zhi H. Factors Influencing Students’ Continuance Usage Intention with Virtual Classroom during the COVID-19 Pandemic: An Empirical Study. Sustainability. 2023; 15(5):4420. https://doi.org/10.3390/su15054420

Chicago/Turabian Style

Huang, Xin, and Hui Zhi. 2023. "Factors Influencing Students’ Continuance Usage Intention with Virtual Classroom during the COVID-19 Pandemic: An Empirical Study" Sustainability 15, no. 5: 4420. https://doi.org/10.3390/su15054420

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop