*Article* **Optimizing the Systematic Characteristics of Online Learning Systems to Enhance the Continuance Intention of Chinese College Students**

**Mengfan Li \* , Ting Wang , Wei Lu and Mengke Wang**

**\*** Correspondence: limengfan0824@163.com

**Abstract:** Different from systems that directly provide online shared courses such as MOOC, online learning systems such as Tencent Classroom simulate a real classroom environment for students and teachers to realize online face-to-face teaching, utilized during the COVID-19 pandemic. Nevertheless, due to the limitation of physical distance, the intelligent design of online learning systems is necessary to provide students with a good learning experience. This study notes that an unexpected optimization effect is the impact of system characteristics on the flow experience of online learning systems, which has not been studied, but plays a vital role in the effectiveness of online learning systems. In the study, a questionnaire was created and multi-stage sampling was used to investigate 623 college students. Based on the DeLone and McLean model of IS success and flow theory, a model for optimizing system characteristics and flow experience was constructed and its effectiveness was tested. The results reveal that system characteristics have a positive impact on continuance intention and flow experience. Additionally, flow experience and learning effect have a positive impact on continuance intention. Furthermore, flow experience has a positive impact on the learning effect. This study emphasizes the flow experience of online learning systems and reveals the optimization direction of online virtual face-to-face classrooms to provide references for the Ministry of Education, schools, and enterprises providing education systems.

**Keywords:** online virtual classrooms; distance learning; flow experience; interactive system; education

### **1. Introduction**

Since 2020, the COVID-19 epidemic has spread rapidly around the world, posing a serious threat to human life and health. Due to the severe situation of COVID-19 and urgent prevention and control policies, universities all over the world have canceled offline courses in favor of online teaching. Before that, online courses were not widely valued in China, while they were common abroad, particularly in the United States [1,2]. Although Chinese enterprises in the field of science, technology, and education have been committed to developing new technologies to improve online learning systems and create a better online learning environment in recent years, this has not been fully utilized by schools. To prevent the spread of COVID-19, China took rigorous prevention and control measures. China's Ministry of Education has instructed schools to delay the start of school and turn to online teaching, creating an excellent opportunity for online learning systems to flourish in China.

Online learning systems are important teaching tools to support intelligent online education. Different from students with high school education or below, who mainly study recorded lessons, college students and professors use online learning systems for online face-to-face live teaching because they can freely use electronic devices and some professional courses need to be taught face-to-face [3]. The biggest difference between this new teaching mode and the traditional teaching mode is that face-to-face classrooms are moved online to enable teachers and students to interact in real-time although they are

**Citation:** Li, M.; Wang, T.; Lu, W.; Wang, M. Optimizing the Systematic Characteristics of Online Learning Systems to Enhance the Continuance Intention of Chinese College Students. *Sustainability* **2022**, *14*, 11774. https://doi.org/10.3390/ su141811774

Academic Editor: Hao-Chiang Koong Lin

Received: 5 August 2022 Accepted: 19 September 2022 Published: 19 September 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

School of Economics and Management, Communication University of China, Beijing 100024, China

not in the same classroom space [4,5]. This teaching mode needs the support of systems, such as Tencent Classroom, Rain Classroom, and TronClass, which are the most important online learning systems providing virtual face-to-face classrooms in China. These systems pay more attention to creating a virtual online classroom for students and teachers as a functional tool to provide online communication scenarios, unlike those systems that offer online shared courses directly, such as MOOC and ZHIDAO. In contrast, it is obvious that the former has a better real-time interaction function to simulate traditional offline courses [6], and is more suitable for sustainable distance education, especially for courses with higher professionalism in universities [7]. It can be seen that it is vital to explore online learning systems providing virtual face-to-face classrooms, which are conducive to long-term online learning. These special online learning systems that provide online virtual face-to-face classrooms can be called online virtual classrooms.

Online virtual classrooms can be regarded as generalized information systems, where teachers and students transfer course information at both ends of the system [8]. Moreover, as information systems, they have the functions of processing and storing course information. For example, teachers can access statistics on students' classroom performance, and students can play back courseware. It means that the systems based on this functional design can surpass the traditional offline courses in the whole teaching experience [9,10]. It is puzzling that college students' continuance intention to use these online virtual classrooms is not high [11]. Some studies explored the influencing factors of college students' willingness to use online learning systems based on the expectation-confirmation model (ECM) and analyzed the impact of factors such as users' subjective satisfaction [12,13]. Some scholars pointed out that technical characteristics play an important role in many factors that affect the users' willingness to use the information system continuously [14–17]. However, in the field of online learning, there is limited research on college students' continuance intention to use online virtual classrooms based on system characteristics. The DeLone and McLean model of IS success (D&M model) can better examine users' continuance intention that directly shows the optimization effect of the system from system quality, information quality and service quality [18]. Therefore, it is meaningful to explore the influencing factors of college students' continuance intention to use online virtual classrooms based on the D&M model.

As a state of user experience, flow experience is closely related to the continuous use of information system users, and significantly affects the users' information system adoption and use behavior [19,20]. These online learning systems rely on the internet, so the delay and instability of the network can more or less affect classroom performance, which is also a reflection of the system quality [21]. Bao [22] pointed out that teachers generally needed to slow down when using online learning systems for teaching, which easily destroyed the immersive input of teachers and students in class, resulting in a poor overall teaching effect. In the teaching process, the use of "flow theory" can effectively stimulate students' interest in learning and improve their learning motivation, which is worth paying attention to in online learning [23]. These systems pay little attention to flow experience in current system designs, but more attention to the system infrastructure. It is of great significance to focus on the flow experience in the subsequent optimization of system characteristics, which will guide the optimization direction of system characteristics to better enhance the continuance intention of college students. In addition, Hashemi's [24] survey found that there was a strong positive correlation between students' learning effect and their satisfaction with online learning. After obviously feeling that the learning effect of using online learning systems is not good, college students will be dissatisfied with online virtual face-to-face classrooms and will not continue to use them [25]. It can be seen that flow experience and learning effect are two important factors that cannot be ignored when exploring college students' willingness to continue using online virtual classrooms. It is a pity that there is no research on them at the same time, and our research can make up for this.

This study aims to explore the general mechanism of college students' intention to use online virtual classrooms. Based on the D&M model, this study focused on the mediation effect of flow experience and the learning effect and tested the hypotheses through the survey data. The results of this study will provide important suggestions for the Ministry of Education, schools, and enterprises providing systems, and explain the optimization direction of online virtual face-to-face classrooms providing simulated offline classes in the future.

### **2. Literature Review**

### *2.1. The COVID-19 Pandemic and Online Learning Systems Providing Virtual Classrooms*

Due to COVID-19, China's Ministry of Education launched an initiative entitled "Ensuring learning undisrupted when classes are disrupted". In response to this directive, Chinese schools have to teach online, especially for universities with complex student sources. Online learning systems are used as functional tools to provide virtual face-to-face classrooms for college students first forced long-term online learning.

In universities, the courses of different majors vary greatly. This poses a challenge for the use of online learning platforms in universities. Husár and Dupláková [26] explored the possibility of online education as a tool to improve language education. Pather et al. [27] summarized the rapid development of online teaching after the outbreak of the pandemic in management anatomy education and found that it was helpful to realize the synchronous teaching of remote sites. Kang and Seo [28] investigated the basic practical methods of online physics teaching and revealed that online learning systems provided more development opportunities. In the study of industrial majors, further optimization can be achieved by using e-learning [29]. These studies support a view that online learning systems can provide online virtual learning for many different majors in universities. This can also reassure different kinds of universities to use online learning systems during the COVID-19 pandemic, which further demonstrates the importance and applicability of these systems.

At present, the research topics of such emerging online learning systems are relatively concentrated. Most studies analyzed the teaching mode [30], technical application [31], and teaching effect [32] of these systems from the perspective of cases. Yu [33] adopted a hybrid design to collect quantitative and qualitative data, and the comparison showed that the availability of using online learning systems for online learning as an aid to learning is significantly higher than other methods. For universities, these systems simulate the traditional offline classroom and pay attention to adding interactive functions such as raising hands, group discussions, and quizzes, to restore the classroom to the greatest extent. During the COVID-19 pandemic, the value of these systems that provide online virtual face-to-face classrooms is self-evident, but it is necessary to investigate in depth the general mechanism of students' continuance intention to use them. Therefore, this study focused on online virtual classrooms that are different from previous online learning systems.

Tencent Classroom has main functions and can be used as a representative of online virtual classrooms. An interactive interface of the class is shown in Figure 1. At the bottom of Figure 1 is the function bar including raising hands, answering questions and so on. On its right is the chat box, where you can receive and send messages. The button above the chat box can be used to invite students into the classroom. These designs ensure that students and teachers can interact promptly.

### *2.2. Online Virtual Classrooms and the DeLone and McLean Model of IS Success*

The D&M model focuses on the three dimensions of information systems: how system quality, information quality, and service quality affect user satisfaction, use intention, and user income [34]. Some scholars used the D&M model for reference to pay attention to the impact of the three elements on users' willingness to continue using the information system [14–16]. Previous studies mostly used the ECM, technology acceptance model (TAM), and other models to investigate users' continuance intention to use online virtual classrooms. Only a few studies began to pay attention to the impact of the three elements in the D&M model in recent years and made a preliminary exploration [35].


**Figure 1.** An Example of classroom interface in Tencent Classroom. **Figure 1.** An Example of classroom interface in Tencent Classroom.

*Sustainability* **2022**, *14*, x FOR PEER REVIEW 4 of 18

*2.2. Online Virtual Classrooms and the DeLone and McLean Model of IS Success* The D&M model focuses on the three dimensions of information systems: how system quality, information quality, and service quality affect user satisfaction, use intention, and user income [34]. Some scholars used the D&M model for reference to pay attention to the impact of the three elements on users' willingness to continue using the information system [14–16]. Previous studies mostly used the ECM, technology acceptance model (TAM), and other models to investigate users' continuance intention to use online virtual classrooms. Only a few studies began to pay attention to the impact of the three elements in the D&M model in recent years and made a preliminary exploration [35]. System availability is one of the important characteristics of system quality, which affects users' willingness to continue using these services [36]. In addition, information interaction in information systems is very important, especially for online virtual classrooms [11]. Cheng et al. [37] revealed that the timeliness of information feedback had a positive impact on users' continuance intention. When conducting online learning, teachers need to transfer course information to students through online learning systems, such as some quizzes and course tasks. These systems provide feedback for the course information input by teachers to students by displaying it on the screen or uploading it to the learning database provided by systems. This can also be called "information feedback" of online learning systems. Similar to other information systems, feedback timeliness of online learning systems may have a positive effect on the willingness of college students to continue using online learning systems. In terms of information quality, timeliness, completeness and intelligibility are important features [38]. Given the motivation of students in the use of online virtual classrooms to obtain knowledge, these characteristics are not as important as the richness and interest of teaching content. The "richness" of course content refers to the breadth, depth, and meaning of it. The "interestingness" of course content depends on ingenious and flexible teaching methods to stimulate students' interest in learning, which is conducive to their understanding of the course content. A study suggested that the richness and interestingness of course content in online virtual classrooms greatly influenced learners' willingness to attend classes [39]. Therefore, it is necessary to focus on how the content interest in information quality affects college students' continuance intention to use online virtual classrooms. Moreover, the service quality of information systems significantly affects user satisfaction [40]. The current online virtual classrooms optimize the quality of service by increasing the personalized function of the information system, System availability is one of the important characteristics of system quality, which affects users' willingness to continue using these services [36]. In addition, information interaction in information systems is very important, especially for online virtual classrooms [11]. Cheng et al. [37] revealed that the timeliness of information feedback had a positive impact on users' continuance intention. When conducting online learning, teachers need to transfer course information to students through online learning systems, such as some quizzes and course tasks. These systems provide feedback for the course information input by teachers to students by displaying it on the screen or uploading it to the learning database provided by systems. This can also be called "information feedback" of online learning systems. Similar to other information systems, feedback timeliness of online learning systems may have a positive effect on the willingness of college students to continue using online learning systems. In terms of information quality, timeliness, completeness and intelligibility are important features [38]. Given the motivation of students in the use of online virtual classrooms to obtain knowledge, these characteristics are not as important as the richness and interest of teaching content. The "richness" of course content refers to the breadth, depth, and meaning of it. The "interestingness" of course content depends on ingenious and flexible teaching methods to stimulate students' interest in learning, which is conducive to their understanding of the course content. A study suggested that the richness and interestingness of course content in online virtual classrooms greatly influenced learners' willingness to attend classes [39]. Therefore, it is necessary to focus on how the content interest in information quality affects college students' continuance intention to use online virtual classrooms. Moreover, the service quality of information systems significantly affects user satisfaction [40]. The current online virtual classrooms optimize the quality of service by increasing the personalized function of the information system, with the focus on the interaction between students and teachers, such as raising hands, bullet screens, and group discussions [41]. These systems use these functions to simulate offline scenes as much as possible to bring a more real face-to-face classroom experience. Meanwhile, the BOPPPS teaching model points out that participatory interaction in students' online learning can effectively improve the classroom experience and teaching effect [42]. Students will be satisfied with a good classroom experience and are willing to continue to use online virtual classrooms [43]. Based on the above analysis, the following hypotheses were proposed:

**H1.** *System availability has a positive effect on college students' continuance intention to use online virtual classrooms*.

**H2.** *Feedback timeliness has a positive effect on college students' continuance intention to use online virtual classrooms*.

**H3.** *Interesting content has a positive effect on college students' continuance intention to use online virtual classrooms*.

**H4.** *System functionality has a positive effect on college students' continuance intention to use online virtual classrooms*.

**H5.** *Interactive sociality has a positive effect on college students' continuance intention to use online virtual classrooms*.

### *2.3. Flow Theory, Learning Effect, and Continued Intention*

Flow experience is a positive psychological experience, which will give individuals such a great sense of pleasure when participating in activities that it urges individuals to repeat the same activities without getting tired [44]. With the continuous development of information technology, various types of user-oriented information systems have been launched. These information systems are designed from the perspective of user experience, and the sense of immersion is one of the important experience feelings. Therefore, the phenomenon of information users' flow experience and the impact of flow experience on information behavior was confirmed by many studies [45–47]. Online virtual classrooms also pay more attention to the application of new technologies to optimize the system interface and user experience to provide a flow experience, which can attract and retain users. The optimization direction of online learning systems is worth exploring.

Moneta and Csikszentmihalyi [48] pointed out that the relative balance between challenges and skills was an important factor in generating flow experience according to flow theory. Human–computer interaction refers to the technology that people use to realize the dialogue between people and computers in an effective way through computer input and output devices [49]. In the context of human-computer interaction, the interface usability design of information systems has an important impact on the balance of challenge skills. High system availability can reduce users' burden, and then achieve the balance between challenges and skills in the process of human-computer interaction, which is conducive to the sense of immersion [50]. At the same time, the feedback mechanism is an important condition for generating a flow experience. When the system interface of online virtual classrooms runs smoothly and quickly responds to user requests and timely feedback on the results required by users, users may be able to gain a flow experience in continuous interaction [51]. In terms of information quality characteristics, Shin confirmed that more attractive content acted on the sense of immersion [52]. In addition, online learning is prone to loneliness and distraction, so it is necessary to enhance the interaction between teachers and students to break the physical distance, to improve the sense of immersion [25]. Ma and Li [7] found that new functions of online virtual classrooms, such as bullet screens, group discussions and chat boxes, could meet the needs of communication between teachers and students, and also ensure students' immersive learning.

Individuals in flow experience can obtain pleasure and satisfaction that cannot be obtained in daily life [53]. Relevant studies found that flow experience has a significant positive impact on the continuance intention of information users [54]. A poor flow experience will make students dissatisfied with online virtual classrooms and unwilling to use online virtual classrooms [25]. Therefore, the following hypotheses were proposed:

**H6.** *System availability has a positive effect on the flow experience*.

**H7.** *Feedback timeliness has a positive effect on the flow experience*.

**H8.** *Interesting content has a positive effect on the flow experience*.

**H9.** *System functionality has a positive effect on the flow experience*.

**H10.** *Interactive sociality has a positive effect on the flow experience*.

**H11.** *Flow experience plays a mediating role in the influence mechanism of the five system characteristics on continuance intention to use online virtual classrooms*.

Due to the differences in individual psychology and behavior, students' acceptance ability in classroom learning is different [55]. This leads to differences in their level of knowledge acquired in class and their performance in completing the test. These different degrees of performance are taken as the main measures of learning effect in this study. It should be noted that online learning makes it impossible to communicate face-to-face directly. It is very easy for the technical characteristics of the system, such as an unstable network and delayed message reception, to keep up with the teaching progress and show poor learning results [56]. It can be seen that the system characteristics of online learning systems, such as system availability, feedback timeliness, have a positive effect on the learning effect. A study showed that the poor flow experience of online learning was more likely to make students feel anxious, affect their classroom performance, and achieve poor learning results [57]. However, no matter what causes the learning effect, students are extremely dissatisfied with online virtual classrooms after they perceive that using online virtual classrooms will weaken their academic performance, and then they become unwilling to continue to use online virtual classrooms [24,25]. Therefore, we hypothesized that:

**H12.** *System availability has a positive effect on the learning effect*.

**H13.** *Feedback timeliness has a positive effect on the learning effect*.

**H14.** *Interesting content has a positive effect on the learning effect*.

**H15.** *System functionality has a positive effect on the learning effect*.

**H16.** *Interactive sociality has a positive effect on the learning effect*.

**H17.** *Flow experience has a positive effect on the learning effect*.

**H18.** *Learning effect plays a mediating role in the influence mechanism of the five system characteristics on continuance intention to use online virtual classrooms*. *Sustainability* **2022**, *14*, x FOR PEER REVIEW 7 of 18

> The research model in this paper is based on the D&M model and flow theory [58,59]. Based on the above hypotheses, this research model as shown in Figure 2. The research model in this paper is based on the D&M model and flow theory [58,59]. Based on the above hypotheses, this research model as shown in Figure 2.

**Figure 2.** Proposed research model. **Figure 2.** Proposed research model.

**3. Materials and Methods**

*3.1. Questionnaire items*

*3.2. The Research Setting*

Medical University and Central University of Finance and Economics.

and confirmed before the questionnaire was issued.

English translations of the questionnaire to ensure accuracy and consistency.

modified by a pre-investigation test. We also repeatedly compared the Chinese and

Since undergraduates in Beijing used online virtual classrooms during the COVID-19 pandemic, with a high length of time and a high proportion of participation, this study limited the survey objects to undergraduates in Beijing, which is very representative in China. To obtain the most economical and effective results under the premise of fully representative samples, we adopted the multi-level sampling method to gradually sample from the types of universities, specific universities, and students. The results of each level are shown in Table 1. In addition, all participants were informed

In stage 1, in order to investigate more comprehensive types of students, we first adopted PPS sampling to select school types. Before sampling, the size of the type sample box was set as 6 university types. We used random numbers to select them according to the number of undergraduates in each university. In stage 2, schools are selected by stratified sampling. One school was selected randomly from each of the six selected school types. In stage 3, simple random sampling was used to randomly select undergraduates from Beijing University of Posts and Telecommunications, Beijing Foreign Studies University, Peking University, Capital Normal University, Capital

### **3. Materials and Methods**

### *3.1. Questionnaire Items*

After an extensive review of the literature on the D&M model, we developed the measurement items (as shown in Appendix A). This scale was reviewed by experts and modified by a pre-investigation test. We also repeatedly compared the Chinese and English translations of the questionnaire to ensure accuracy and consistency.

### *3.2. The Research Setting*

Since undergraduates in Beijing used online virtual classrooms during the COVID-19 pandemic, with a high length of time and a high proportion of participation, this study limited the survey objects to undergraduates in Beijing, which is very representative in China. To obtain the most economical and effective results under the premise of fully representative samples, we adopted the multi-level sampling method to gradually sample from the types of universities, specific universities, and students. The results of each level are shown in Table 1. In addition, all participants were informed and confirmed before the questionnaire was issued.


**Table 1.** Sampling results of types of colleges and universities in the first stage.

In stage 1, in order to investigate more comprehensive types of students, we first adopted PPS sampling to select school types. Before sampling, the size of the type sample box was set as 6 university types. We used random numbers to select them according to the number of undergraduates in each university. In stage 2, schools are selected by stratified sampling. One school was selected randomly from each of the six selected school types. In stage 3, simple random sampling was used to randomly select undergraduates from Beijing University of Posts and Telecommunications, Beijing Foreign Studies University, Peking University, Capital Normal University, Capital Medical University and Central University of Finance and Economics.

### *3.3. Description of the Sample*

Due to COVID-19, we issued questionnaires online and offline to ensure the greatest accuracy of sampling and quality of questionnaires, and 716 questionnaires were collected. According to the answers to the audit questions set in the questionnaire, the length of time to fill in, the identification of reverse questions, and other judgment factors, the invalid questionnaires were eliminated item by item. Finally, 623 valid questionnaires were used for analysis with an effective rate of 87.01%. Table 2 shows the demographic characteristics of the participants.

**4. Results** 

carried out.

*4.1. Measurement Model* 

Feedback timeliness (FT)

Interesting content (IC)

System functionality (SF)

Interactive sociality (IS)

Flow experience (FE)

Learning effect (LE)


**Table 2.** Demographic characteristics.

Since the issuance of questionnaires strictly followed the sampling design, the distribution of the final valid samples was also very appropriate. The results show that 270 (43.34%) are male and 353 (56.66%) are female. The respondents are mainly in grade 2 (272, 43.82%) and grade 3 (191, 30.50%). The benefit from taking into account the school type in the sampling design, respondents' professional types cover thirteen categories. Since students of different majors may need different functions and tools to match their major content when using online virtual classrooms [60–62], a full investigation of students in various majors can avoid bias caused by professional imbalance when examining students' functional requirements. Science 74 11.88 History 6 0.96 Agronomy 11 1.77 Literature 92 14.77 Medical Science 74 11.88 Art 21 3.37 Philosophy 8 1.28 **4. Results**  Military Science 6 0.96 Science 74 11.88 History 6 0.96 Agronomy 11 1.77 Literature 92 14.77 Medical Science 74 11.88 Art 21 3.37 Philosophy 8 1.28

Military Science 6 0.96

#### **4. Results** *4.1. Measurement Model*

#### *4.1. Measurement Model* To test common method bias [63,64], the Harman single factor test was used to

To test common method bias [63,64], the Harman single factor test was used to analyze all the measurement items of each latent variable. The explained percentage of the variance of the first common factor is 31.857%, which is lower than the critical value of 40%, indicating that the common method bias is within an acceptable range. It can be considered that there is no serious common method bias and further research can be carried out. analyze all the measurement items of each latent variable. The explained percentage of the variance of the first common factor is 31.857%, which is lower than the critical value of 40%, indicating that the common method bias is within an acceptable range. It can be considered that there is no serious common method bias and further research can be carried out. To test common method bias [63,64], the Harman single factor test was used to analyze all the measurement items of each latent variable. The explained percentage of the variance of the first common factor is 31.857%, which is lower than the critical value of 40%, indicating that the common method bias is within an acceptable range. It can be

Next, we tested the reliability and effectiveness. Cronbach's Next, we tested the reliability and effectiveness. Cronbach's ɑ values and composite reliability (CR) values were used as indicators to test the effectiveness of variables. In Table 3, the Cronbach's ɑ values of all variables are higher than 0.8, and the composite reliability (CR) values are higher than 0.7. Therefore, the scale established in this study has good internal consistency and high reliability. values and composite reliability (CR) values were used as indicators to test the effectiveness of variables. In Table 3, the Cronbach's considered that there is no serious common method bias and further research can be Next, we tested the reliability and effectiveness. Cronbach's ɑ values and composite reliability (CR) values were used as indicators to test the effectiveness of variables. In Table 3, the Cronbach's ɑ values of all variables are higher than 0.8, and the composite values of all variables are higher than 0.8, and the composite reliability (CR) values are higher than 0.7. Therefore, the scale established in this study has good internal consistency and high reliability.

**Table 3.** Reliability and validity test. reliability (CR) values are higher than 0.7. Therefore, the scale established in this study **Table 3.** Reliability and validity test.

(FT)

FT1 0.909

FT3 0.855

IC1 0.943

IC3 0.872

SF1 0.833

SF3 0.716

IS1 0.741

IS3 0.733 IS4 0.889

FE1 0.944

FE3 0.744 FE4 0.759

LE1 0.830

LE3 0.795

Continuance intention CI1 0.909 0.884 0.719 0.879

Interesting content (IC)

FT2 0.894 0.913 0.779 0.909

IC2 0.787 0.908 0.769 0.899

SF2 0.857 0.850 0.657 0.842

0.884 0.658 0.879 IS2 0.857

0.856 0.664 0.869 FE2 0.739

LE2 0.821 0.889 0.680 0.856

System functionality (SF)

Interactive sociality (IS)

Flow experience (FE)

Learning effect (LE)


IC2 0.787 0.908 0.769 0.899

SF2 0.857 0.850 0.657 0.842

0.884 0.658 0.879 IS2 0.857

0.856 0.664 0.869 FE2 0.739

LE2 0.821 0.889 0.680 0.856

FT3 0.855

IC1 0.943

SF1 0.833

SF3 0.716

IS1 0.741

IS3 0.733 IS4 0.889

FE1 0.944

FE3 0.744 FE4 0.759

LE1 0.830

LE3 0.795

Continuance intention CI1 0.909 0.884 0.719 0.879


**Table 3.** *Cont.*

KMO = 0.921, Bartlett spherical approximate chi-square test value is 10,709.622, *p* < 0.001.

In addition, the factor loadings and average variance extracted (AVE) values of all variable measures are higher than 0.5, indicating that the scale has good convergent validity. Meanwhile, in Table 4, the square root of AVE is significantly greater than the correlation coefficient of each variable, so the scale has good discrimination effectiveness. Finally, the results of confirmatory factor analysis show that: χ <sup>2</sup>/df = 2.076 < 3.0, GFI = 0.948 > 0.9, CFI = 0.972 > 0.9, NFI = 0.948 > 0.9, NNFI = 0.967 > 0.9, RMSEA = 0.043 < 0.1, RMR = 0.098 < 0.1, all goodness of fit indicators are within the recommended range, which shows that the model fits well.


**Table 4.** The square root of AVE and correlation coefficient of variables.

### *4.2. Structural Equation Modeling*

In this study, SEM was used to test the theoretical model and hypotheses, whose strength lies in the quantitative study of the interaction between multiple variables [58,65]. The path coefficient indicates the degree of support of the data to the research model. Figure 3 shows the SEM test results of the sample. It can be seen from Figure 3 that only system availability (β = 0.160, *p* < 0.001) and interactive sociality (β = 0.199, *p* < 0.001) have significant positive effects on college students' continuance intention to use online virtual classrooms. H1 and H5 are supported, while H2–H4 are not tenable. In addition, system availability (β = 0.359, *p* < 0.05), feedback timeliness (β = 0.210, *p* < 0.001), interesting content (β = 0.138, *p* < 0.001), system functionality (β = 0.188, *p* < 0.001) and interactive

sociality (β = 0.154, p < 0.001) have a positive effect on flow experience. H6–H10 are verified. At the same time, feedback timeliness (β = 0.183, *p* < 0.001), interesting content (β = 0.216, *p* < 0.001), system functionality (β = 0.224, *p* < 0.001) and interactive sociality (β = 0.156, *p* < 0.001) have a significant positive effect on the learning effect. However, system availability (β = 0.065, *p* = 0.077 > 0.05) has no significant effect on the learning effect, while it has a tremendous impact on flow experience. H13–H16 are proved, but H12 is not. *Sustainability* **2022**, *14*, x FOR PEER REVIEW 11 of 18 positive effect on the learning effect. It is also obvious that the flow experience of students plays an important role in the entire influence mechanism.

**Figure 3.** Path coefficient of the model. Note: All coefficients in the above figure are standardized coefficients; the solid line and the dashed line represent the significant and non-significant effects of the corresponding path, respectively; \*\* *p* < 0.001. **Figure 3.** Path coefficient of the model. Note: All coefficients in the above figure are standardized coefficients; the solid line and the dashed line represent the significant and non-significant effects of the corresponding path, respectively; \*\* *p* < 0.001.

*4.3. Mediation Analysis* The theoretical model shows that there is a chained mediating relationship between system characteristics, flow experience, learning effect and continuance intention. To Finally, flow experience (β = 0.273, *p* < 0.001) and learning effect (β = 0.292, *p* < 0.001) have a significant positive effect on college students' continuance intention to use online virtual classrooms. Flow experience (β = 0.152, *p* < 0.001) has a significant positive effect on the learning effect. H11 is verified.

further verify the mediating role of flow experience and learning effect between system characteristics and college students' continuance intention to use online virtual classrooms, a bootstrap sampling test (5000 samples) was conducted using the SPSS macro program PROCESS plug-in compiled by Hayes [66]. The results are shown in Table 5 and Table 6. First, interactive sociality was studied. In Table 5, the interactive sociality (β = 0.155, *p* < 0.05) positively affects college students' continuance intention to use online virtual classrooms. Moreover, interactive sociality has significant positive effects on flow To sum up, the five system characteristics of the study have a positive effect on college students' continuance intention to use online virtual classrooms, of which interactive sociality has the greatest impact. Except that the system availability has no obvious effect on the learning effect, these system characteristics indirectly affect the college students' continuance intention to use online virtual classrooms by positively affecting the flow experience and learning effect. At the same time, flow experience has a positive effect on the learning effect. It is also obvious that the flow experience of students plays an important role in the entire influence mechanism.

### experience (β = 0.155, *p* < 0.01) and learning effect (β = 0.176, *p* < 0.01). Based on model M3, flow experience and learning effect were introduced into models M4, M5 and M6. It is *4.3. Mediation Analysis*

found that the influence of interactive sociality, flow experience and learning effect on continuance intention is still evident, demonstrating that there is a partial mediation. In addition, Table 6 also shows a chained mediation between interactive sociality, flow experience, learning effect, and continuance intention (95% confidence interval is [0.001, 0.015]). The results of the mediating effect analysis are provided in Table 6. Among them, The theoretical model shows that there is a chained mediating relationship between system characteristics, flow experience, learning effect and continuance intention. To further verify the mediating role of flow experience and learning effect between system characteristics and college students' continuance intention to use online virtual classrooms, a bootstrap sampling test (5000 samples) was conducted using the SPSS macro program PROCESS plug-in compiled by Hayes [66]. The results are shown in Tables 5 and 6.

the 95% interval (BootCI) values of the indirect effect do not include the number 0, indicating that it has a mediating effect. The final results show that only the learning effect has no mediating effect between system availability and continuance intention, while the mediating effect between other variables in the research model and hypothesis exists. Through the above analysis, H11 is supported and H18 is partially supported.


### **Table 5.** Regression analysis of mediating effect.

Note: \* *p* < 0.05 \*\* *p* < 0.01.

**Table 6.** Indirect effect analysis results.


Note: BootLLCI refers to the lower limit of the 95% interval of bootstrap sampling, and BootULCI refers to the upper limit of the 95% interval of bootstrap sampling. These grey parts are chained mediation and the rest are parallel mediation.

First, interactive sociality was studied. In Table 5, the interactive sociality (β = 0.155, *p* < 0.05) positively affects college students' continuance intention to use online virtual classrooms. Moreover, interactive sociality has significant positive effects on flow experience (β = 0.155, *p* < 0.01) and learning effect (β = 0.176, *p* < 0.01). Based on model M3, flow experience and learning effect were introduced into models M4, M5 and M6. It is found that the influence of interactive sociality, flow experience and learning effect on continuance intention is still evident, demonstrating that there is a partial mediation. In addition, Table 6 also shows a chained mediation between interactive sociality, flow experience, learning effect, and continuance intention (95% confidence interval is [0.001, 0.015]).

The results of the mediating effect analysis are provided in Table 6. Among them, the 95% interval (BootCI) values of the indirect effect do not include the number 0, indicating that it has a mediating effect. The final results show that only the learning effect has no mediating effect between system availability and continuance intention, while the mediating effect between other variables in the research model and hypothesis exists. Through the above analysis, H11 is supported and H18 is partially supported.

### **5. Discussions**

### *5.1. Theoretical Implications*

This study combined the D&M model and flow theory to investigate the generation mechanism of Chinese college students' continuance intention to use online virtual classrooms. The results of this study can make some theoretical contributions to future research.

First, most of the previous research on online learning systems focused on systems that provide shared online courses. They paid more attention to gameplay and course design when exploring users' continuance intention [14], and never paid attention to the flow experience. However, a study by Simamora et al., suggested that poor flow experience in learning is more likely to make students feel anxious and affect their classroom performance and willingness to attend class [56]. Although the impact of flow experience on users' willingness to use information systems has been confirmed by many studies, this aspect has not been taken into account by previous studies on online learning systems to the best of the author's knowledge. This study innovatively examines systems such as Tencent Classroom, which provide real-time online virtual classrooms for universities during the COVID-19 pandemic and expands the important role of flow experience through the D&M model. The results obtained in this study are consistent with the conclusion of Simamora et al., [56]. Second, previous research on online learning systems has commonly been conducted in conjunction with course performance [67,68]. It is undeniable that compared with course performance, the learning effect recognized through quizzes during and after class has a more intuitive impact on students' use intention and the learning effect is affected by flow experience [24]. The influence of flow experience on the learning effect is incorporated into the model, which improves the generation mechanism of the whole college students' continuance intention to use online virtual classrooms.

### *5.2. Practical Implications*

There is an old saying in China: "a workman must first sharpen his tools if he is to do his work well." Therefore, it is more noteworthy to optimize online virtual classrooms to improve users' satisfaction and willingness to continue using them. Compared with a study investigated the way flow experience drives the online students' intention to engage in online English teaching platforms based on the flow and expectation confirmation model (ECM), which pays more attention to students' perception [69], this study applies D&M model to optimize system characteristics from the perspective of applicability and can provide more practical optimization suggestions.

This study identifies that the flow experience of college students plays an important role in the whole influence mechanism. College students with a better flow experience will show better learning effects and higher continuance intention to use online virtual classrooms. Online virtual classrooms need to pay attention to the application of new technologies to optimize the system and user experience to provide an immersive learning experience to attract and retain users. The existing online learning systems have provided basic system support and interactive functions, allowing teachers and students to communicate at both ends of the system. However, it can be seen that the flow experience it provides is not satisfactory from the feedback of teachers and students. Previous studies have shown that functions such as bullet screen, line dropping prevention, and screen cutting processing performed well in providing an immersive learning experience [70]. The survey results of this paper also indicate that system availability and feedback timeliness have a great positive impact on the flow experience. The system side can improve these. For example, these systems can optimize the system performance to adapt to more application scenarios and reduce reflection time. It can reduce errors in the system and transfer tasks assigned by the teacher to the students in a shorter time. In addition, the design of the system interface can also be more scientific, such as the clear division of modules, so that

students can enter and exit the course and find information more conveniently. These will make students more accessible to use these systems to achieve a similar or even better flow experience than offline, which will optimize their learning effect and continuance intention.

This study also finds that interesting content and system functionality have a great impact on the learning effect. This reminds teachers to pay more attention to the design of courseware during online teaching, which is also a function that the system can focus on. Moreover, it is found that the functions of recording and playback courses and sharing courseware are also worth developing and optimizing to enrich the usability of the system before, during, and after class. In addition, the interaction between teachers and students is an essential part of online courses, which can effectively reduce the sense of distance and loneliness to drive students' learning mood. Online virtual classrooms can design personalized bullet screen functions and various discussion spaces to arouse students' enthusiasm for interaction.

### *5.3. Inadequate Research and Future Recommendations*

Although the sampling design of this study is scientific and rigorous, only relatively representative college students in Beijing were investigated because of the sampling difficulties during the COVID-19 pandemic. As there may be some local policies in different regions that will affect the online learning experience of college students, the generalizability of this research conclusion needs to be strengthened and the follow-up research can further expand the scope of the investigation. Moreover, although the new structure and project were created by combining important literatures and checked repeatedly before the formal investigation, the implementation of each key point in the questionnaire could be further optimized through focus group interviews [71]. Moreover, we did not consider control variables in this study. Future research may examine the effects of control variables, such as demographic variables and personality traits; they could influence the findings [58]. More importantly, without COVID-19, online learning is often only used as an auxiliary tool rather than the main learning method [33,72]. In the future, we can compare the different effects of college students, using it both as an auxiliary tool and the main learning method.

### **6. Conclusions**

The prevention and control of COVID-19 is a battle of resistance and difficulty. We may need to make great efforts to overcome it, but learning cannot be delayed. Online learning systems simulate the real classroom environment for college students during the COVID-19 pandemic to achieve online face-to-face teaching as much as possible. However, from a large number of blog posts on social media, college students feel widespread dissatisfaction with online courses, which reminds us that we need further analysis and optimization to enhance the experience of future online learning. Since the topic and direction of this study are relatively new and there is no directly available scale, this study developed the measurement items to investigate the generation mechanism of Chinese college students' continuance intention to use online virtual classrooms. Based on the D&M model, this study explored the relationship between flow experience, learning effect, and college students' continuance intention to use online virtual classrooms.

In this study, it is found that system characteristics are very important for online virtual classrooms to retain college students. The flow experience of college students plays an important mediating role in the whole shock mechanism. According to the investigation of this study, after considering the flow experience, these systems will be optimized in technology and function, such as increasing the personalized design of information interaction and technology to prevent screen cutting. The optimization of these system characteristics will greatly enhance the flow experience of college students, thus increasing their willingness to continue using the system. This study emphasizes the flow experience of online learning systems and reveals the optimization direction of online virtual face-to-face classrooms to provide references for the Ministry of Education, schools, and enterprises providing systems for education.

**Author Contributions:** Conceptualization, M.L., T.W. and W.L.; data curation, M.L. and M.W.; investigation, M.L.; methodology, M.L. and W.L.; software, M.L.; supervision, M.L. and T.W.; validation, M.L. and T.W.; visualization, M.L. and M.W.; writing—original draft, M.L. and T.W.; writing—review and editing, M.L. and W.L. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Informed consent was obtained from all subjects involved in the study.

**Data Availability Statement:** Data can be available from the corresponding author upon request.

**Acknowledgments:** The authors wish to thank the reviewers for their invaluable comments and suggestions that enhanced the quality of the paper.

**Conflicts of Interest:** The authors declare no conflict of interest.

### **Appendix A**

**Table A1.** Questionnaire items.


### **References**

