Next Article in Journal
CO2 Emission and Cost Optimization of Concrete-Filled Steel Tubular (CFST) Columns Using Metaheuristic Algorithms
Next Article in Special Issue
Training, Education, and Research in COVID-19 Times: Innovative Methodological Approaches, Best Practices, and Case Studies
Previous Article in Journal
Spatial Analysis of Mangrove Forest Management to Reduce Air Temperature and CO2 Emissions
Previous Article in Special Issue
Cyber-Physical System of Psychophysiological Support of Professional Self-Realization in Professions of the ‘Man-Nature’ Type in the Formation of Specialists for Sustainable Development
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Sustainability
Volume 13
Issue 14
10.3390/su13148089
sustainability-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

The Critical Factors of Student Performance in MOOCs for Sustainable Education: A Case of Chinese Universities

by
Qin Yang
1 and
Young-Chan Lee
2,*
1
College of Economics, Jiaxing University, Jiaxing 314001, China
2
Department of Business Administration, College of Management and Economics, Dongguk University, Gyeongju 38066, Korea
*
Author to whom correspondence should be addressed.
Sustainability 2021, 13(14), 8089; https://doi.org/10.3390/su13148089
Submission received: 30 May 2021 / Revised: 10 July 2021 / Accepted: 13 July 2021 / Published: 20 July 2021
(This article belongs to the Special Issue Training, Education and Research in COVID-19 Times: Innovative Methodological Approaches, Best Practices, and Case Studies)
Browse Figures
Versions Notes

Abstract

:
COVID-19 has drastically changed human life, both in the current era and, no doubt, in the years to come. One of the hot topics during this period is whether and how COVID-19 will change education patterns in the next few years. This study aimed to investigate the relationship between massive open online courses (MOOCs) and students’ learning performance. To that end, this study combined the gamification concept with the modified information systems (IS) success model and the expectation–confirmation model (ECM) for building a research model. We surveyed 586 students through an online survey and tested the hypotheses. The research results indicated that MOOCs’ information quality (IQ), system quality (SQ), and service quality (SEQ) had positive influences on the confirmation, which then affected MOOCs’ usefulness, satisfaction, and gamification. Students’ continued usage intention and course performance were positively influenced by usefulness and gamification. Students’ continued usage intention of MOOCs also positively affected course performance. The relationship between gamification and satisfaction, however, was not proved. The research results also showed that gamification, which was composed of entertainment, challenge, and social interaction, played a critical role in improving students’ continued usage intention of MOOCs and their course performance.

1. Introduction

The outbreak of COVID-19 spread worldwide at an unprecedented rate, becoming the first global pandemic over the past 100 years, causing severe harm and threatening human lives. However, the impacts and damages of the pandemic on the economy, politics, culture, education, and ideology remain unassessed. At the beginning of 2020, for the first time in history, worldwide universities closed offline classes as an emergency measure to prevent the increase in the virus infection rate. Thanks to the massive open online course (MOOC), hundreds of millions of students worldwide can continue their studies. Universities could work in a usual fashion during the lockdown period by providing temporary access to learning on MOOC platforms. During the pandemic, Chinese universities had to organize teaching and learning online. In December of 2020, Tsinghua University and the UNESCO Institute for Information Technologies in Education co-hosted the Global MOOC Conference. The minister of the Ministry of Education of the People’s Republic of China (MOE) pointed out that MOOCs in China started in 2013, after 7 years of rapid development, with more than 32,000 courses online. The number of courses and the application scale currently rank first among countries in the world. Leading Chinese universities created and offered MOOCs, and students from other universities could enroll in these courses. Teachers evaluated available courses and chose the most appropriate course for students first. Typically, each course lasts 11–14 weeks. Students must pass quizzes and a final exam to receive a certificate from the MOOC provider (e.g., https://www.icourse163.org/ accessed on 21 June 2021) and eventually obtain credit from their university. Teachers usually provide extra instruction and consultation as a supplement to ensure students’ learning outcomes.
Education institutions have made tremendous efforts, and teachers have created online courses and delivered them to students through the Internet. The “virtual semester” has started in many universities, and various classes are launched and provided online in a well-organized manner. In the field of education, many countries and regions, including China, have accumulated experience in the past efforts and exploration to combat COVID-19 outbreaks, but many unprecedented problems, meanwhile, have been revealed. Humans are bound to coexist with COVID-19 for a long time to come. In order to restore and rebuild the ecosystem of education, maintain the vitality of the education system, and promote the healthy and sustainable development of education in the post-pandemic era, it is not only necessary to reflect on and examine the experience accumulated in the fight against COVID-19 but also to conduct systematic and comprehensive plans from the perspective of education sustainability in the future. As an important mode of online education, MOOC gradually occupies an important position in traditional education, has its own unique advantages, and tends to take over the sole dominance of traditional education. MOOC has emerged from the rapid development of technology and the urgent need for education [1]. The MOOC’s distinguishing feature is its convenience and extensive sharing, making it a tool for improving the quality of education with high efficiency and proper education at a low cost. In addition to recorded lectures and assignments, many MOOCs provide tools that enable students to interact with the teachers and communicate with online classmates through social media and online communities. MOOCs can also provide immediate feedback, online examinations, and assignments. Interaction between students and professors can occur within an online community through synchronous and asynchronous online learning activities, and students’ social, interactive, and collaborative skills will improve [2]. More than this, students can select their schedule for studying with the assistance of their teacher. After COVID-19, MOOC has been widely discussed in academic and practical areas as an innovative tool for sustainable education.
There is a debate because people wonder whether online classes offered by schools will be attractive to students. With the outbreak of COVID-19, most of our daily activities have been transferred from offline to online. Many educational institutions worldwide provide MOOCs, including edX, Moodle, CourseSites, Udemy, Versal, etc. According to Class Central, a search engine and reviews site for free online courses popularly known as MOOCs, by 2019, MOOCs had reached 110 million learners. Most importantly, the top three MOOC platforms (Coursera, edX, and FutureLearn) registered as many new learners in April 2020 as in the whole of 2019. Worldwide platforms provide more than 2500 courses, 11 online degrees, and 170 micro-credentials. The use of MOOCs during COVID-19 has allowed educational institutions to ensure teaching work runs smoothly on the rails. It enables students to have greater autonomy in preparing and conducting their learning. MOOCs differ significantly in terms of their purposes and the mechanisms that power them. Main categories of MOOCs usually include: First, connectivist MOOCs (cMOOCs) are a relatively loose online community for learners, mainly focusing on a shared learning interest. cMOOCs, on the other hand, use various social media tools and personal interaction tools for learning and sharing knowledge among the participants. Second, eXtended MOOCs (xMOOCs) are additional supplementary courses at universities. Third, cooperative MOOCs often combine characteristics of both xMOOCs and cMOOCs. Lastly, gamification cooperative MOOCs (gcMOOCs) [3,4] add gamification designs to the cooperative MOOCs.
The focus of previous research on MOOCs has been varied. Some key topics include barriers and intentions for using MOOCs [5], MOOC scholarship [6], challenges and motivation for teachers and students [7], and engagement and retention [8]. Lee et al. [9] discussed MOOCs from the perspective of self-regulated learning. Furthermore, many studies have discussed the ECM in explaining and exploring learners’ continued usage intentions. For instance, Dai et al. [1] extended the ECM and proposed a theoretical model including cognitive and affective variables to discuss and explain users’ intentions to stay in MOOC learning. Zhou [10] added “social influence” as a new construct to the original ECM in the MOOCs’ learning context.
Gamification is a prevalent concept in mobile and technological systems that adopts a game mechanism to facilitate desired behaviors and bring positive performance outcomes. It has been proven that gamification in MOOCs positively influences students’ motivation [11]. Among the diverse research theories that were used for discussing MOOCs, the gamification theory was a notable one. Antonaci et al. [12] provided a gamification design process with six phases. In designing gamified MOOCs, learner, social, openness, course design, and technology were considered. Antonaci et al. [13] summarized the effects of game elements on learners’ behavior in an online learning environment, such as performance, motivation, engagement, collaboration, and social awareness. Dominguez et al. [14] and Dicheva et al. [15] indicated that gamification of online learning could increase users’ motivation for studying if adequately designed and implemented. Thus, gamification is a favorable mechanism that can motivate users of information systems. Malone [16] emphasized the concept of gamification. This study focused on stimulating more people to consider game-based elements in education because game mechanisms influence users’ learning motivation and engagement. As gamification gains increasing attention in education research, many scholars have emphasized the benefits that gamified designs bring to learners. Their findings provide breakthroughs for future studies in online learning and gamification research [17,18,19]. In order to maintain and increase students’ interest and engagement, the gamified design of MOOCs should include emotional incentives by giving students challenging cases, entertaining functions, and social interaction opportunities. Although those studies noted that gamification affected students’ positive experience with the MOOC, to the best knowledge of the authors, no study has been arranged to examine the relationship between gamification and overall effects of MOOCs. To address this research gap, the current study adopted an integrated framework based on the IS success model, expectation–confirmation model (ECM), and gamification concept to investigate the relationship between MOOCs and students’ course performance empirically. For the analysis and research hypothesis tests, we used covariance-based structural equation modeling (CB-SEM). This paper is organized as follows. The description of MOOC and the theoretical basis of this study are systematically reviewed and arranged in Section 2.1, Section 2.2, Section 2.3, Section 2.4 and Section 2.5. Then we propose a research model and hypotheses in Section 2.6. Next, we describe the data collection and research methods in Section 2.7 and Section 2.8, followed by data analysis results in Section 3. Finally, we discuss the major findings, implications, and limitations in the final part of the paper.

2. Materials and Methods

2.1. MOOCs (Massive Open Online Courses)

The media used in MOOCs is open to the public. Learners are encouraged to share and contribute materials. Course modules are often 5–10 min, and course content is edited when needed. As to traditional online courses, most institutions have their own online course platform and restrict the media. Students search for courses within an institution’s library database. Course modules are typically 45–60 min long. In MOOCs, lectures are usually pre-recorded, and courses are open-ended with no due dates. As for traditional online courses, in addition to recorded lectures, the instructors sometimes conduct live lectures. Courses frequently end on the due date. The learners of MOOC include not only school students but also on-job students. As with traditional online courses, learners are often students of the institution. In addition, unlike online courses, which focus more on content, MOOCs focus more on context.
Wu and Chen [20] suggested that MOOCs are attractive to students who have strong motivation and who hold the perception that MOOCs are useful for individual users. To further examine and discuss the factors influencing students’ perceptions and intentions to use MOOCs, many studies have paid attention to the antecedents of learners’ behavioral intentions from various perspectives [5,10,21,22]. Although past research has successfully revealed various valuable and interesting implications and provided views that support the importance of an actively engaged MOOC, it is necessary to investigate MOOCs’ effect on students’ continued usage intentions and course performance.

2.2. Overview of a Chinese MOOC Platform

In MOOCs, each course runs periodically, and the entire learning process consists of multiple sessions: attending lectures, participating in discussions, submitting assignments, pop quizzes, and a final exam. Courses are managed and operated by the Academic Affairs Office of each university. Universities create courses and designate the teachers to manage the courses.
Usually, there are 9 sections involved in one course, from course selection to certificate issuance, as shown in Figure 1.
In addition, students should finish authentication first so that teachers can manage the students in an organized manner, as shown in Figure 2:
The design and structure of a course include 4 parts: a bulletin board, courseware (video and PPT files), assignments and quizzes, and a discussion board. Consider the course Financial Risk Management, as shown in Figure 3.
The first figure on the left shows the bulletin board of Financial Risk Management, which includes the course outline, description of each class, and final exam description. The second figure presents the courseware of the course, where students can find lecture videos from chapter 1 to the last chapter and obtain PPT and/or PDF files. The third figure indicates the assignments and quizzes of the course. Students should finish and pass these assignments and quizzes before the deadline. The last figure demonstrates the discussion board of the course. The discussion board is divided into three sections: general discussion, Q&A, and the Learning Exchange Zone. On the discussion board, students can communicate with the teachers and their classmates.

2.3. IS Success Model

The implementation success of an information system can be influenced by a lot of factors, such as environmental factors, system factors, end-users’ characteristics, etc. [23] DeLone and McLean [24,25] proposed the IS success model, an integrated model that comprehensively explains IS success. The model presents, identifies, describes, examines, and explores the relationships between the essential dimensions. For decades, research of information systems and technology has adopted the IS success model. From 1995 to 2019, various studies have applied the IS success model and provided beneficial results [16,26,27,28,29,30].
In online education, well-established research models and theories such as the technology acceptance model (TAM), the theory of reasoned action (TRA), the theory of planned behavior (TPB), and the unified theory of acceptance and use of technology (UTAUT) have been discussed so far. Rogers’ [31] innovation diffusion theory (IDT) was also a remarkable one. In addition, some researchers examined students’ behaviors in online education by employing the IS success model. For example, Freeze et al. [32] and Aparicio et al. [33] investigated e-learning system success using the IS success model. Cidral et al. [34] proposed a research model integrating the IS success model and success factors in e-learning systems. Albelbisi [35] verified the role of quality factors in self-regulated learning in MOOCs, and Albelbisi [36] also developed and validated MOOCs’ success factors using Malaysian samples.
Information quality (IQ), system quality (SQ), and service quality (SEQ) are three independent variables in the IS success model. IQ deals with the supplied information from the information system with accuracy, correlation, and conformance. SQ usually refers to the usefulness, stability, function, reliability, complexity, compatibility, and demonstrability of the IS. SEQ discusses whether information systems provide assured, responsive, and empathetic service. Most previous studies usually included IQ, SQ, SEQ, intention to use, user satisfaction, and net benefits when researching information technology and systems [37,38,39,40,41]. MOOC platform provides content courses, a learning environment, and services for potential learners through the Internet. IQ of the MOOC platform refers to information provided on the MOOC platform having good quality when contents are perceived as useful, reliable, complete, and understandable. SQ of the MOOC platform refers to good system technical performance, accessibility, availability, and usability. SEQ of the MOOC platform refers to the responsiveness, competence, and empathy of the MOOC platform service representatives. From previous studies using these IS success variables (IQ, SQ, SEQ) [39,41,42], we presume that this model is often adopted in a variety of fields since it provides a comprehensive understanding of the context in which the users use the system. Hence, it is also important to explore and examine how these factors influence success on MOOC platforms. MOOC platforms are attracting attention as a new online education tool, and it is important to keep these platforms available to students. This study focused on improving students’ course performance by continuously using the MOOCs platform. We proposed that the information system success model can be adopted since it is useful to measure how students perceive the usefulness and gamified designs of a MOOC platform, students’ satisfaction with a MOOC platform, their continued use intention toward a MOOC platform, and their learning performance resulting from the involvement. MOOCs presuppose the importance of their content, scheme, and instant service as a representative online education tool. These attributes have been neglected in the evaluation of MOOCs. As a result, it is critical to explore and explain MOOCs’ qualities, which may significantly influence learners’ perceptions and behaviors.

2.4. Expectation–Confirmation Model

When considering post-purchase or post-adoption satisfaction, previous studies have used various constructs. Some representative antecedents of continued usage intentions include perceived usefulness, satisfaction, perceived ease of use, trust, etc. Many studies have discussed users’ continued usage intentions of information technologies through the expectation–confirmation model (ECM). Hsu and Lin [43], Leung and Chen [44], and Gupta et al. [45] investigated the relationship between pre-adoption experiences and post-adoption continued usage intentions in the field of paid mobile apps and e-health technology. Wu et al. [40] explored online impulse buying through the expectation–confirmation model, an epistemic research model that explores post-adoption satisfaction and continued usage of the information system. Bhattacherjee [46] demonstrated that expectations formed by accepting a specific information system would lead to confirmation by comparing the characteristics and performance. When users confirm the performance, their satisfaction will be followed. Thus, in turn, continued usage intentions will be induced.

2.5. Gamification

In IS research, gamification is an emerging concept that uses gamified designs and non-gamified technologies to make tasks more engaging. Gamification has a profound impact on many different types of information systems, both in theory and action. British-born computer programmer and inventor Nick Pelling initially proposed the concept of gamification in 2002. It started to gain attention and hit mainstream academic circles around 2010. Research in the health system, crowdsourcing, education, etc., has been applied to gamification. One of the predominant problems of MOOCs is students’ low retention rates and completion rates [47,48]. This phenomenon emphasizes the importance of introducing a gamification mechanism to increase the retention and completion rate. Levels, badges, leaderboards, points, feedback, challenges, likes, and social features, communication channels, narratives, etc., are the most widely used gamified elements in MOOCs [13,33]. Figure 4 shows a “Lucky Card” game provided by a MOOC provider during exam week, and Figure 5 shows a badge.
The Lucky Card game is a “Will Not Fall” game with a lucky meaning. By engaging this game, students can draw different cards randomly. Each card represents a “weapon” for the final examination, such as the “memory bread” card, which means you will be able to memorize all the contents of the exam. “Energy milk” card means you will be full of energy. “Not fall spray” card means you will outdo yourself in the exam. These cards are supposed to be lucky in Chinese culture. Furthermore, students can share this game with their friends on social networks and have various discussions together. These entertaining games can help students reduce the pressure of studying psychologically. In addition to that, students will be rewarded (points gained) when they complete predefined activities of a weekly mission, such as watching video lectures and finishing quizzes. The digital badges are also incentives to motivate students to continue learning.
Dominguez et al. [14] indicated the benefits and consequences of gamified elements and designs in an e-learning platform, and Dicheva et al. [15] prepared systematic mapping research to discuss gamified elements in education. An et al. [49] found that learners had various attitudes concerning the gamification of MOOCs. Academics and industry are becoming more interested in the gamification of MOOCs. What influence does gamified design have on MOOCs? Among the effects, the most remarkable one must be an increase in participation. When students participate in the discussion forum, they have an opportunity to gain “likes” from classmates and teachers. This is a way to increase students’ participation. Furthermore, group interactions are available for students as a tool for getting feedback and making contributions. In addition, time pressure is another gamified element for encouraging students’ participation. Time pressure means MOOC platforms give students a time limit to finish a certain activity to push them to complete a required task. For students, creating time pressure can encourage them to participate in learning positively because the time pressure is connected to their learning goals (e.g., finishing the current learning stage and moving to a higher level). By adding the gamification concept to MOOCs, students’ completion rate will increase, and students will be willing to spend more time on the MOOCs. Ortega-Arranz al. [19] and Khalil et al. [50] showed that students’ retention increased by employing gamification, and their motivation and engagement were improved. Saputro et al. [51] found that students’ intrinsic motivation in MOOC increased under the framework of fulfilling rules and suitability of the instruments and game elements. The current study tried to add the gamification concept to the IS success model and expectation–confirmation model framework based on the previous studies and findings.

2.6. Research Model and Hypotheses

2.6.1. MOOC Qualities and User Confirmation

DeLone and McLean [24] proposed a research model explaining information system success. The original research model includes six constructs, system quality, information quality, use, user satisfaction, and individual and organizational impact. The updated IS success model was proposed by them later in 2003. The updated model [25] added a new construct, service quality, to the original model. In addition, the “use” in the basic model was divided into two separate variables: “intention to use” and “use” for measuring and explaining IS success. In online education, Hassanzadeh et al. [52], and Marjanovic et al. [53] applied the IS success model for exploring the e-learning platform. Lin [54] indicated that the platform’s SQ, IQ, and SEQ significantly influenced actual online learning platform use through users’ satisfaction and intention to use. The ECM is built on consumer behavior studies and integrates theoretical and empirical findings from previous IS literature. Bhattacherjee [55] suggested that users’ continued usage intention (CI) is influenced by their satisfaction with the IS and users’ perception of its usefulness. The users’ dissatisfaction will result in the discontinuance of the IS if performance disconfirms the expectations.
By using MOOCs and evaluating their performance, learners constitute perceptions in their minds and compare MOOCs with other learning tools, corresponding to their expectations. The qualities of MOOCs are connected with potential users’ expectations of MOOCs. The higher the users’ perceived qualities of the MOOCs and confirmed expectations, the more positive perceptions and satisfaction will be induced. The study of Roca et al. [56] proved that the IQ, SQ, and SEQ of an e-learning system positively affected expectation–confirmation of the e-learning system.
Similarly, Chung et al. [57] also suggested that an information system’s confirmation was positively influenced by its IQ, SQ, and design quality (DQ). In addition, in smartphone classes, Shin et al. [58] showed that the system’s perceived content quality and perceived service quality positively influenced users’ confirmation. Based on previous studies, this paper hypothesizes that MOOCs’ information quality (IQ), system quality (SQ), and service quality (SEQ) have positive influence on MOOCs’ quality confirmation:
Hypothesis 1 (H1).
Information quality of the MOOC positively affects students’ confirmation of MOOC qualities.
Hypothesis 2 (H2).
System quality of the MOOC positively affects students’ confirmation of MOOC qualities.
Hypothesis 3 (H3).
Service quality of the MOOC positively affects students’ confirmation of MOOC qualities.

2.6.2. User Confirmation, MOOC Platform Usefulness, User Satisfaction, and Gamification

Bhattacherjee [46] showed that users’ confirmation of their original expectations is a critical predictor of users’ usefulness perception formation. Information system confirmation deals with the degree to which the information systems satisfy the user’s expectations. The perceived usefulness of an information system implies the user’s expectation of that information system, influencing users’ post-adoption behavior. To be more specific, if any user confirms his/her expectations, their perceived usefulness will be formed. In addition, ECM proposed that users’ satisfaction with an information system is induced by their confirmation of experience and the perceived usefulness of the system. Previous studies have proved the relationships between users’ confirmation, perceived usefulness, and users’ satisfaction [1,45,59,60]. Pozón-López et al. [61] showed that satisfaction is affected by MOOC quality and other qualities such as usefulness. In MOOCs, if students find that the actual performance is better than their original expectations, their confirmation of the platform will be positive. In turn, the positive confirmation will result in a higher level of user satisfaction. On the contrary, if students believe that the system’s performance falls short of their expectations, their confirmation will be negative. Consequently, the negative confirmation will lead to a lower level of user satisfaction in the subsequent stages. According to Bhattacherjee and Lin [55], user satisfaction is an emotional evaluation of an information system based on previous experience.
In addition to the above constructs, gamification is a new critical component of the information system. Although gamified elements’ influences have been explored and discussed in several fields, studies on gamification in MOOCs are still limited. Some researchers have recently addressed the relationship between gamification and expected benefits [2,51,62,63]. Extrinsic motivation could be considered and explained in terms of gamified elements in MOOCs. While learning, learners engage in gamified MOOCs to get points and obtain badges and other prizes. Recently, studies discussing the effectiveness of the gamified design of the online educational system have been growing. However, it is necessary and valuable to conduct empirical studies to explore the relationship between gamification (e.g., badges, points, forum participation, “likes”, time pressure, social networking interaction, etc.) and learners’ behaviors and learning performance. Students’ perceptions of gamified elements of the MOOCs are their expectations of the system, and the expectations will influence their post-adoption behaviors. If students’ expectations can be confirmed, the perception of the gamified design will be formed as well. Based on previous gamification literature, we hypothesized that:
Hypothesis 4 (H4).
Students’ confirmation of MOOC qualities positively affects MOOC usefulness.
Hypothesis 5 (H5).
Students’ confirmation of MOOC qualities positively affects MOOC gamification perceptions.
Hypothesis 6 (H6).
Students’ confirmation of MOOC qualities positively affects their satisfaction with the MOOC.
Hypothesis 7 (H7).
MOOC usefulness positively affects students’ satisfaction with the MOOC.
Hypothesis 8 (H8).
MOOC gamification positively affects students’ satisfaction with the MOOC.

2.6.3. MOOC Usefulness, Satisfaction, Gamification, and Continuance Intention

In MOOCs, the platform’s perceived usefulness deals with the degree to which MOOCs can be useful and improve students’ learning. Ajzen [64] believed that users’ subjective perceptions influence their behaviors, even though the perceptions they hold are tendentious or incorrect. As a result, instead of an objective assessment, the perceived feeling is supposed to be relevant. According to Bhattacherjee [46], users’ satisfaction with an information system is an active, neutral, or passive feeling. Expectation–confirmation studies state that users’ satisfaction with an information system is a prerequisite for building users’ continued usage intentions. Bhattacherjee [46], Joo et al. [65], Li and Fang [66], and Bölen and Özen [67] proved the positive relationship between perceived usefulness and IS continued usage intention, as well as satisfaction and IS continued usage intention. Hence, in MOOCs, students are expected to use MOOCs continuously if satisfied with the platform. On the other hand, the direct relationship between gamification and users’ continued usage intentions has not been empirically verified yet. However, several studies have proved that users’ benefits and continued usage will increase in a gamified environment [68,69]. Hence, we hypothesized that students’ perceptions of MOOCs’ gamified design will positively influence their continued usage intentions in the MOOC context.
Hypothesis 9 (H9).
MOOC usefulness positively affects students’ continued usage intention of MOOC.
Hypothesis 10 (H10).
MOOC gamification positively affects students’ continued usage intention of MOOC.
Hypothesis 11 (H11).
Satisfaction with the MOOC positively affects students’ continued usage intention of MOOC.

2.6.4. Usefulness, Gamification, Continued Usage of MOOCs, and Course Performance

Course performance in MOOCs refers to the “perceived outcome” of learning through MOOCs. Specifically, it deals with the ability and possibility to learn well with higher efficiency and less effort and time. Tam and Oliveira [70] investigated individual performance in the context of m-banking. However, insufficient empirical research has studied the salient relationship between the IS’ perceived usefulness and individual performance and the IS’ continued usage intention and individual performance. Usefulness keeps ahead of user satisfaction, and users’ positive experience with usefulness will induce greater user satisfaction. Increased IS user satisfaction leads to increased IS usage intentions, which improves individual performance [71]. Increased use of MOOCs may raise student awareness of the benefits of MOOCs. In a MOOC learning context, students who adopt digital systems and have digital skills can improve their learning performance. This study investigated whether students could gain a clear understanding of the classes through using MOOCs, measured whether students could easily achieve the learning goals asserted by the course via MOOCs, and discussed if it is easier to accomplish the assignments by using MOOCs.
Gamified design is another critical element of MOOCs. Gerdenitsch et al. [72] examined how gamification affects work enjoyment, productivity, and the moderating role of leadership responsibility. Feng et al. [73] discussed whether gamification could increase consumers’ engagement with fitness apps. Groening and Binnewies [74] examined the relationship between digital achievements, motivation, and performance. The research results show that digital achievements could improve motivation and enhance performance. The findings provide significant inspiration for future IS research. MOOCs employ gamified design to induce and increase students’ engagement and participation. A gamified MOOC may change students’ study patterns and communication patterns. Thus, if students feel satisfied with the novel studying tools and the interaction patterns, their motivation and engagement will increase. In turn, students’ learning performance will be improved compared with those online education tools without gamified design. Based on previous studies, we hypothesized that:
Hypothesis 12 (H12).
MOOC usefulness positively affects students’ course performance.
Hypothesis 13 (H13).
MOOC gamification positively affects students’ course performance.
Hypothesis 14 (H14).
Students’ continued usage intention of the MOOC positively affects students’ course performance.

2.6.5. Gamification: Sociality, Entertainment, and Challenges

As discussed above, gamification is a way to increase students’ motivation and engagement and, finally, affect students’ course performance. The gamification concept has also gained attention from researchers and practitioners. Legaki et al. [75] proved that the challenge-based gamified elements in an online learning platform positively affect students’ learning compared to conventional teaching tools. The concept of gamification has been employed in many research areas. For example, Tobon et al. [76] examined the relationship between gamified features and online customer behavior, and Rodrigues et al. [77] explored gamification in the field of e-banking. Specifically, Rodrigues et al. [77] investigated how bank users face gamified smart applications, mainly focused on entertainment and ease of use elements. Their findings reveal that the system’s ease of use and entertainment had an impact on e-banking usage. Social interaction is another critical component of gamification. Fang et al. [21] stated that online learning platforms’ social interaction design positively influences users’ study focus. Students could feel a sense of entirely participating or breathtakingly engaging in the MOOCs’ learning [78]. If applied to MOOCs, social interaction may stimulate frequent use of the platform and possibly improve students’ learning performance. In addition, the social interaction design may improve the students’ MOOC user experience. Without face-to-face communication, students still maintain social connections with professors and classmates. Thus, we propose that gamification may include vibrant designs by providing students with challenges, entertaining atmosphere, and social interaction in MOOCs.
Hypothesis 15 (H15).
Gamification is a second-order construct that is comprised of socialness, challenge, and entertainment.
Based on the hypotheses described above and previous studies, this study presents a research model as shown in Figure 6.

2.7. Data Collection

This study conducted a questionnaire survey to collect data and investigate the relationship between the constructs. After completing an extensive review of IS literature, we created the survey items and translated the original ones into Chinese. Then, we translated them into English again to ensure the questionnaire’s accuracy and consistency. We distributed the survey questionnaires to universities in China and collected 629 questionnaires. All of the students attended MOOCs in the spring semester of 2020. After excluding 43 questionnaires with incomplete responses, 586 valid questionnaires were used for analysis.
The respondents’ demographic characteristics are summarized in Table 1. Table 1 shows 209 (35.7%) males and 377 (64.3%) females. In terms of educational backgrounds, most of those who responded were college students (482, 82.3%). In terms of past MOOC user experience, 136 (23.2%) have used them for less than a month, 127 (21.7%) have learned in them for 1 to 3 months, and 107 (18.3%) have used them for more than 12 months. When it comes to the average amount of time spent on MOOCs each week, most participants (260, 44.4%) spend 1 to 4 h per week on them. Finally, 160 (27.3%) respondents came from the country’s western region, 215 (36.3%) from the country’s central region, and 211 (36%) from the country’s eastern region.

2.8. Measurement Items

The measurement items were developed through an extensive literature review (see Appendix A). The survey aimed at measuring students’ perceptions of MOOCs positively or negatively, which includes the following variables: information quality (IQ), system quality (SQ), service quality (SEQ) of the MOOC platform, students’ confirmation, perceptions of the MOOC platform’s gamified characteristics, perceived usefulness of MOOCs, students’ satisfaction with MOOCs, students’ MOOC platform continued usage intentions, and students’ course performance through learning on MOOC platforms.
This study employed a 5-point Likert scale, ranging from 1 to 5 (1 represents an inclination to strongly disagree, 5 represents strongly agree). We analyzed the collected data in 3 steps. First, outliers, normality, and missing values were checked. Second, we tested the validity and reliability of the measurement items. Finally, we examined correlations between the constructs, tested the model fit, checked common method bias, and tested research hypotheses.

3. Analysis and Results

This study analyzed the data using the covariance-based structural equation model (CB-SEM). The measurement model and structural model were evaluated, respectively.

3.1. Measurement Model

We evaluated the measurement model by checking reliability and convergent and discriminant validity. In Table 2, all values of standardized item loadings for the indicator were ≥0.7. All of the composite reliability (CR) values were greater than 0.8. All values of Cronbach’s α for each construct were ≥0.8. All average variance extracted (AVE) values employed for assessing the convergent validity were ≥0.5. Consequently, Table 2 indicates that all values met the minimum requirement, and the measurement model has good reliability and convergent validity.
Pearson’s correlation coefficients are shown in Table 3. Next, we compared the square root of AVE with the variables’ correlation coefficients to assess the discriminant validity. As shown in Table 3, the AVE’s square root is significantly larger than each variable’s correlation coefficient. Consequently, the scale has good discriminant validity.
Podsakoff and Organ [79] and Podsakoff et al. [80] suggested that there were possibilities for common method variance (CMV) in the self-reported data. We applied the Harman one-factor statistical analysis to examine the common method bias in the collected data. First, we checked if the measurements were affected by common method variance through Harman’s single factor test and examined whether relationships between constructs were observed to inflate or deflate. This test includes a factor analysis of all the involved items. In addition, we performed exploratory factor analysis without rotation and checked whether all factors were returned. Previous research suggested that if one factor’s total variance exceeds 50%, it shows common method bias in the study. However, as shown in Table 4, one factor explained only 27.899% of the total variance. Thus, we can conclude that there is no common method bias in the collected data.
Finally, we compared the recommended values and actual values of model fitness. Table 5 shows the values of χ2, degrees of freedom (df), goodness-of-fit index (GFI), adjusted goodness-of-fit index (AGFI), normed fit index (NFI), comparative fit index (CFI), parsimony goodness fit index (PGFI), root mean square residual (RMR), and root mean square of approximation (RMSEA). All actual values met the standard of recommended values, which indicated a good model fit.

3.2. Structural Model

This section examined the R2 of MOOC platform confirmation, usefulness, satisfaction, continued usage intention, course performance, gamification (social interaction, entertainment, and challenge), which suggested the explanatory power of the independent variables. The path coefficients and R2 showed how well the data supported the proposed research model. It is recommended that the R2 value of 75%, 50%, and 25% may indicate the substantial, moderate, and weak level of the construct’s explanation power (s).
Information quality (β = 0.312 and p-value < 0.001), system quality (β = 0.172 and p-value = 0.001), and service quality (β = 0.217 and p-value < 0.001) have significant positive effects on MOOC quality confirmation. Three qualities explained MOOC quality confirmation with R2 = 33.8%. Thus, Hypotheses 1, 2, and 3 are supported. MOOC quality confirmation significantly positively affected MOOC usefulness (β = 0.489 and p-value < 0.001) with R2 = 23.9%. Thus, Hypothesis 4 is supported. MOOC quality confirmation significantly positively affected MOOC gamification perceptions (β = 0.463 and p-value < 0.001) with R2 = 21.5%. Therefore, Hypothesis 5 is supported. In addition, MOOC quality confirmation (β = 0.453 and p-value < 0.001) and MOOC usefulness (β = 0.375 and p-value < 0.001) have significant positive effects on students’ satisfaction with MOOCs (R2 = 55.3%). However, gamification did not have a statistically significant effect on satisfaction. Thus, Hypothesis 6 and Hypothesis 7 are supported, but Hypothesis 8 is not. Regarding students’ continued usage intention of the MOOCs, MOOC usefulness (β = 0.23 and p-value < 0.001), MOOC gamification perception (β = 0.365 and p-value < 0.001), and MOOC satisfaction (β = 0.335 and p-value < 0.001) have significant positive effects on students’ continued usage intention (R2 = 51.8%). Therefore, Hypotheses 9, 10, and 11 are supported. In addition, MOOC usefulness (β = 0.275 and p-value < 0.001), MOOC gamification perception (β = 0.392 and p-value < 0.001), and MOOC continued usage intention (β = 0.331 and p-value < 0.001) significantly affected course performance with R2 = 62.1%. As a result, Hypotheses 12, 13, and 14 are supported. Lastly, as a second-order construct, gamification explained 67.5% of the variation of social interaction, 53.3% of the challenge, and 55.3% of entertainment. To summarize, based on these findings, gamification was proven to be a reflective second-order construct of social interaction, challenge, and entertainment. Thus, Hypothesis 15 is supported. Details can be found in Table 6 and Figure 7.

4. Discussion

Campus closure is an important intervention that helps safeguard public health and respond to the spread of epidemic infections. According to the Policy Brief: Education during COVID-19 and beyond provided by the United Nations, the COVID-19 pandemic has created the largest disruption of education systems in history, affecting nearly 1.6 billion learners in more than 190 countries all continents. Closures of campuses and other learning spaces have impacted 94% of the world’s student population, up to 99% in under-developed countries. The sudden outbreak of COVID-19 triggered a global education crisis and, at the same time, forced governments and schools to take the initiative to change education. In response to the ongoing situation of COVID-19, many countries have adopted a variety of flexible teaching and learning approaches in their education systems, among which MOOC is one of the main approaches. The MOOC, as a subset of distance education, has been focused on providing a more flexible and innovative educational experience in terms of time and space than school education by utilizing various types of technology. During the epidemic prevention and lockdown period, many countries closed schools and changed to MOOC mode and thereby accelerated the process of informatization and modernization of education.
The COVID-19 epidemic will not be eliminated for a long period in the future. Human beings are forced to move into and survive in the post-pandemic era with the coronavirus. The way of production, living, and survival also needs to be changed accordingly. Education is crucial to the sustainable development of human society. We have a mission and a task to adapt to coexist with the epidemic, work toward its improvement, and build a better world for future generations. Education in the digital information era is critical to the future development of the quality of human life and the world’s sustainability. New digital technologies in education have far-reaching effects on formal education and informal learning regarding educational goals and objectives, educational ecology and the environment, learning processes, teaching and learning processes, and educational governance and policies.
We surveyed university students who had taken massive open online courses (MOOCs) during the COVID-19 pandemic to investigate the relationship between MOOCs and students’ course performance. All hypotheses were supported except Hypothesis 8 (gamification satisfaction). It indicates that, among the three MOOC platform qualities, the MOOC platform’s information quality was the most critical factor in affecting the quality confirmation of the MOOC platform. In addition, the MOOC platform’s quality confirmation significantly influenced students’ perceived usefulness of the MOOC platform, satisfaction with MOOCs and students’ perceived gamified features. Moreover, only when students perceive MOOCs’ usefulness and gamification, and when they are satisfied with the MOOC platform will they consider using MOOCs for future studies. Furthermore, with students’ perceived usefulness and gamified MOOCs’ features increasing, their learning performance will improve significantly. Finally, if students intend to use MOOCs for their future learning, their learning performance will also considerably improve. The results indicate that students’ continued use intention and learning improved a positive attitude toward MOOCs. It is worth mentioning that we combined gamification with the IS success model and ECM in this study. The findings support the notion that the gamification of MOOCs consists of social interaction, entertainment, and challenges. Furthermore, gamification has a significant impact on students’ intentions to continue using MOOCs and improve course performance.

4.1. Theoretical Implications

The results of this study could provide future studies with some theoretical implications. Firstly, our study utilized an integrated framework combined with DeLone and McLean’s [24,25] IS success model and Bhattacherjee’s ECM [17] and introduced MOOCs’ gamified designs to the research model. Next, we introduced information quality, system quality, and service quality to define MOOC platform qualities. Then, we investigated the overall MOOC platform quality confirmation through the ECM framework. The integrated framework ensures that the MOOC platform quality confirmation links students’ perceived characteristics of the MOOC platform, students’ post usage intentions, and learning performance.
Secondly, most previous research on MOOCs has primarily focused on the users’ intention to adopt or the users’ intention to re-adopt. This study expanded the research model by syncing the MOOC platform’s continued usage intention and course performance and empirically testing the influence of MOOCs’ cognitive perceptions on course performance. The results show that MOOCs significantly impacted students’ course performance through various factors. This study measured the characteristics of MOOCs and users’ behaviors. Another difference between this study and previous studies is that it attempted to consider MOOCs as a learning tool and a combination of technology and a learning option during COVID-19. To the best of the authors’ knowledge, this study may be novel research in measuring how MOOCs influence students’ course performance under the context of a public health emergency.

4.2. Practical Implications

This study also makes several contributions to MOOC platform managers about how to improve their platforms and services. First, the results indicate that MOOC platform qualities had significant influences on the MOOC platform quality confirmation. Information quality had the most significant effect, followed by service quality and system quality. From this perspective, we concluded that the information quality of the MOOC platforms exceeded students’ expectations. For this reason, MOOC platform managers should improve information quality continuously. MOOC platform managers should pay attention to online service or call center service construction as MOOCs’ service quality is lower than students’ expectations. One of the reasons for this phenomenon might be found in the increasing number of users during COVID-19, which meant that users could not get consulting services in a timely fashion. The results also indicate that system quality significantly influences the confirmation of a slight estimate. This emphasizes the importance of system quality in the management of the MOOC platform. MOOC platforms are frequently paralyzed during COVID-19 because tens of millions of students attend online classes at the same time. Improvements to the MOOC platform’s quality will increase students’ pre-expectations.
Second, the results show that students’ perceived usefulness and confirmation of the MOOC platform significantly influenced students’ satisfaction with the MOOC platform. If students believe MOOCs are simple to grasp, useful, well-managed, and beneficial to their studies, it will influence the level of students’ satisfaction with MOOC platforms. MOOC platform managers should improve and upgrade the system and update course materials to provide a better user experience. By increasing the usefulness of MOOC platforms, students will be more likely to use MOOCs for continuous learning. Furthermore, useful course materials, well-constructed content, and an easily navigated system will significantly improve students’ course performance.
Third, the results prove that MOOCs’ continued usage intentions positively influenced students’ course performance. Thus, MOOC platform managers should develop mechanisms to stimulate students’ course completion rates. Lastly, the results indicate that gamification was composed of social interaction, entertainment, and challenges. Gamified mechanisms significantly influence students’ continued usage intentions and their course performance. During COVID-19, students have to take all of their classes online and spend 6~8 h per day on online learning. Therefore, it is necessary to make MOOCs more exciting and attractive to students. As previous research has shown, one of the primary issues with MOOC learning is the low completion rate of students [81]. Gamification mechanisms such as redeemable points, team leaderboards, badges, peer grading, emoticon feedback, peer interaction, etc., will increase the course completion rate. In addition to the current gamified features, MOOC managers should develop other attractive and useful gamified designs to improve students’ MOOCs’ continued usage intentions and course performance.

5. Conclusions

Despite the theoretical and practical implications, this study has some limitations, as follows. First, this study’s results and implications should not be generalized because the survey was conducted in China only. We consider that comparative research in another cultural context could contribute more effectively in the future. Second, we developed a questionnaire based on previous studies and obtained empirical data from online learners. Personal interviews or focus group interviews may be chosen for future studies to create new constructs and items. Third, we did not consider control variables in this study. Future research may examine the influences of control variables such as demographic variables and personality traits, which might influence the research results. Lastly, this study showed that gamification did not significantly influence students’ satisfaction with MOOCs. However, previous studies have reported that gamification in IS is a critical attribute to users’ satisfaction. Thus, we need to conduct further investigations into the relationship between gamification and satisfaction in future studies.

Author Contributions

Conceptualization, Q.Y. and Y.-C.L.; methodology, Q.Y. and Y.-C.L.; software, Q.Y.; validation, Q.Y. and Y.-C.L.; formal analysis, Q.Y. and Y.-C.L.; investigation, Q.Y.; data curation, Q.Y. and Y.-C.L.; writing—original draft preparation, Q.Y.; writing—review and editing, Y.-C.L.; visualization, Q.Y. and Y.-C.L.; supervision, Y.-C.L. Both authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Data Availability Statement

The data presented in this study are available upon request from the corresponding author. Data are not publicly available due to a confidentiality agreement with participants.

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A

Questionnaire items.
FactorsItemsSources
Information quality (IQ)The MOOC platform provides useful information.
The information provided by MOOC platform is understandable.
The information provided by MOOC platform is interesting.
The information provided by MOOC platform is reliable.		[17]
System quality (SQ)The MOOC platform is easy to navigate.
The MOOC platform allows me to find easily the information I am looking for.
The MOOC platform is well structured.
The MOOC platform is easy to use.
Service quality (SEQ)The support staff is always highly willing to help whenever I need support with the MOOC platform.
The support staff provides personal attention when I experience problems with the MOOC platform.
The support staff provides services related to the MOOC platform at the promised time.
The support staff has sufficient knowledge to answer my questions in respect of the MOOC platform.
MOOC quality confirmation (MCON)My experience with using MOOC platform is better than I expected.
The service level provided by MOOC platform is better than I expected.
Content on the MOOC platform is better than I expected.		[57]
MOOC usefulness (MUSE)Using the MOOC platform can improve my study performance.
Using the MOOC platform can increase my study effectiveness.
I think the MOOC platform is useful to me		[56]
MOOC satisfaction (MSAT)I am satisfied with the performance of the MOOC platform.
I am pleased with the experience of using the MOOC platform.
My decision to use the MOOC platform is a wise one.
Social interaction (SOC)I open up more to others via the MOOC than in other communication modes.
I have a network of friends I made via studying through MOOC.
Studying through MOOC enables me to connect with friends in my real life.
Studying through MOOC enables me to keep in touch with friends in my real life.		[82]
Challenge (CHA)The MOOC platform provides “hints” in text that helps me overcome the challenges.
The MOOC platform provides “online support” that helps me overcome the challenges.
The MOOC platform provides video or audio auxiliaries that help me overcome the challenges.		[17]
Entertainment (ENT)For an online education website, MOOC features and applications are funny.
For an online education website, MOOC features and applications are thrilling.
For an online education website, MOOC features and applications are exciting.
For an online education website, MOOC features and applications are delightful.		[83]
MOOC continued usage intention (MCI)I will use the MOOC platform on a regular basis in the future.
I will frequently use the MOOC platform in the future.
I will strongly recommend my friends to use MOOC platform.		[56]
Course performance (CP)I have gained a clear understanding about the classes through using MOOC platform.
I can easily achieve the learning goals asserted by this course via MOOC platform.
By using MOOC platform, it is easier to accomplish the assignments.
I am capable in learning how to make good use of MOOC platform.		[84]

References

  1. Dai, H.M.; Teo, T.; Rappa, N.A.; Huang, F. Explaining Chinese university students’ continuance learning intention in the MOOC setting: A modified expectation confirmation model perspective. Comput. Educ. 2020, 150, 1–16. [Google Scholar] [CrossRef]
  2. Zainuddin, Z.; Chu, S.K.W.; Shujahat, M.; Perera, C.J. The impact of gamification on learning and instruction: A systematic review of empirical evidence. Educ. Res. Rev. 2020, 30, 100326. [Google Scholar] [CrossRef]
  3. Jarnac de Freitas, M.; Mira da Silva, M. Systematic literature review about gamification in MOOCs. Open Learn. J. Open. Dis. eLearn. 2020, 1–23. [Google Scholar] [CrossRef]
  4. Martínez-Núñez, M.; Fidalgo-Blanco, Á.; Borrás-Gené, O. New challenges for the motivation and learning in engineering education using gamification in MOOC. Int. J. Eng. Educ. 2016, 32, 501–512. [Google Scholar]
  5. Mohan, M.M.; Upadhyaya, P.; Pillai, K.R. Intention and barriers to use MOOCs: An investigation among the post graduate students in India. Educ. Inf. Technol. 2020, 25, 5017–5031. [Google Scholar] [CrossRef]
  6. Ebben, M.; Murphy, J.S. Unpacking MOOC scholarly discourse: A review of nascent MOOC scholarship. Learn. Media Technol. 2014, 39, 328–345. [Google Scholar] [CrossRef]
  7. Hew, K.F.; Cheung, W.S. Students’ and instructors’ use of massive open online courses (MOOCs): Motivations and challenges. Educ. Res. Rev. 2014, 12, 45–58. [Google Scholar] [CrossRef]
  8. Paton, R.M.; Fluck, A.E.; Scanlan, J.D. Engagement and retention in VET MOOCs and online courses: A systematic review of literature from 2013 to 2017. Comput. Educ. 2018, 125, 191–201. [Google Scholar] [CrossRef]
  9. Lee, D.; Watson, S.L.; Watson, W.R. Systematic literature review on self-regulated learning in massive open online courses. Australas. J. Educ. Technol. 2019, 35, 28–41. [Google Scholar] [CrossRef]
  10. Zhou, J. Exploring the factors affecting learners’ continuance intention of MOOCs for online collaborative learning: An extended ECM perspective. Australas. J. Educ. Technol. 2017, 33, 123–135. [Google Scholar] [CrossRef] [Green Version]
  11. Barata, G.; Gama, S.; Jorge, J.; Gonçalves, D. Studying student differentiation in gamified education: A long-term study. Comput Hum. Behav. 2017, 71, 550–585. [Google Scholar] [CrossRef]
  12. Antonaci, A.; Klemke, R.; Kreijns, K.; Specht, M. Get Gamification of MOOC right! Int. J. Serious Games 2018, 5, 61–78. [Google Scholar] [CrossRef]
  13. Antonaci, A.; Klemke, R.; Specht, M. The effects of gamification in online learning environments: A systematic literature review. Informatics 2019, 6, 32. [Google Scholar] [CrossRef] [Green Version]
  14. Domínguez, A.; Saenz-De-Navarrete, J.; De-Marcos, L.; Fernández-Sanz, L.; Pagés, C.; Martínez-Herráiz, J.J. Gamifying learning experiences: Practical implications and outcomes. Comput. Educ. 2013, 63, 380–392. [Google Scholar] [CrossRef]
  15. Dicheva, D.; Dichev, C.; Agre, G.; Angelova, G. Gamification in education: A systematic mapping study. J. Educ. Technol. Soc. 2015, 18, 75–88. [Google Scholar]
  16. Malone, T.W. September. What makes things fun to learn? Heuristics for designing instructional computer games. In Proceedings of the 3rd ACM SIGSMALL symposium and the first SIGPC symposium on Small systems, New York, NY, USA, 18–19 September 1980; pp. 162–169. [Google Scholar]
  17. Aparicio, M.; Oliveira, T.; Bacao, F.; Painho, M. Gamification: A key determinant of massive open online course (MOOC) success. Inf. Manag. 2019, 56, 39–54. [Google Scholar] [CrossRef] [Green Version]
  18. Chang, J.W.; Wei, H.Y. Exploring engaging gamification mechanics in massive online open courses. Educ. Technol. Soc. 2016, 19, 177–203. [Google Scholar]
  19. Ortega-Arranz, A.; Bote-Lorenzo, M.L.; Asensio-Pérez, J.I.; Martínez-Monés, A.; Gómez-Sánchez, E.; Dimitriadis, Y. To reward and beyond: Analyzing the effect of reward-based strategies in a MOOC. Comput. Educ. 2019, 142, 103639. [Google Scholar] [CrossRef] [Green Version]
  20. Wu, B.; Chen, X. Continuance intention to use MOOCs: Integrating the technology acceptance model (TAM) and task technology fit (TTF) model. Comput. Hum. Behav. 2017, 67, 221–232. [Google Scholar] [CrossRef]
  21. Fang, J.; Tang, L.; Yang, J.; Peng, M. Social interaction in MOOCs: The mediating effects of immersive experience and psychological needs satisfaction. Telemat. Inform. 2019, 39, 75–91. [Google Scholar] [CrossRef]
  22. Sanchez-Gordon, S.; Luján-Mora, S. Research challenges in accessible MOOCs: A systematic literature review 2008–2016. Univers. Access Inf. Soc. 2018, 17, 775–789. [Google Scholar] [CrossRef]
  23. Yoon, Y.; Guimaraes, T.; O’Neal, Q. Exploring the factors associated with expert systems success. MIS Q. 1995, 19, 83–106. [Google Scholar] [CrossRef] [Green Version]
  24. DeLone, W.H.; McLean, E.R. Information systems success: The quest for the dependent variable. Inf. Syst. Res. 1992, 3, 60–95. [Google Scholar] [CrossRef] [Green Version]
  25. Delone, W.H.; McLean, E.R. The DeLone and McLean model of information systems success: A ten-year update. J. Manag. Inf. Syst. 2003, 19, 9–30. [Google Scholar]
  26. Jang, J.H.; Kim, J.K.; Hwang, Y.H. Influence of hotel information system quality on system use and user satisfaction. J. Qual. Assur. Hosp. Tour. 2006, 7, 41–58. [Google Scholar] [CrossRef]
  27. Lin, H.F.; Lee, G.G. Determinants of success for online communities: An empirical study. Behav. Inf. Technol. 2006, 25, 479–488. [Google Scholar] [CrossRef]
  28. Seddon, P.; Kiew, M.Y. A partial test and development of DeLone and McLean’s model of IS success. Australas. J. Inf. Syst. 1996, 4. [Google Scholar] [CrossRef] [Green Version]
  29. Sharma, S.K.; Sharma, M. Examining the role of trust and quality dimensions in the actual usage of mobile banking services: An empirical investigation. Int. J. Inf. Manag. 2019, 44, 65–75. [Google Scholar] [CrossRef]
  30. Wang, W.T.; Lai, Y.J. Examining the adoption of KMS in organizations from an integrated perspective of technology, individual, and organization. Comput. Hum. Behav. 2014, 38, 55–67. [Google Scholar] [CrossRef]
  31. Rogers, E.M. Diffusion of Innovations, 5th ed.; Free Press: New York, NY, USA, 2003. [Google Scholar]
  32. Freeze, R.D.; Alshare, K.A.; Lane, P.L.; Wen, H.J. IS success model in e-learning context based on students’ perceptions. J. Inf. Syst. Educ. 2010, 21, 173–184. [Google Scholar]
  33. Aparicio, M.; Bacao, F.; Oliveira, T. Grit in the path to e-learning success. Comput. Hum. Behav. 2017, 66, 388–399. [Google Scholar] [CrossRef]
  34. Cidral, W.A.; Oliveira, T.; Di Felice, M.; Aparicio, M. E-learning success determinants: Brazilian empirical study. Comput. Educ. 2018, 122, 273–290. [Google Scholar] [CrossRef] [Green Version]
  35. Albelbisi, N.A. The role of quality factors in supporting self-regulated learning (SRL) skills in MOOC environment. Educ. Inf. Technol. 2019, 24, 1681–1698. [Google Scholar] [CrossRef]
  36. Albelbisi, N.A. Development and validation of the MOOC success scale (MOOC-SS). Educ. Inf. Technol. 2020, 25, 4535–4555. [Google Scholar] [CrossRef]
  37. Isaac, O.; Abdullah, Z.; Ramayah, T.; Mutahar, A.M. Factors determining user satisfaction of internet usage among public sector employees in Yemen. Int. J. Technol. Learn. Innov. Dev. 2018, 10, 37–68. [Google Scholar] [CrossRef]
  38. Urbach, N.; Smolnik, S.; Riempp, G. An empirical investigation of employee portal success. J. Strateg. Inf. Syst. 2010, 19, 184–206. [Google Scholar] [CrossRef]
  39. Wang, Y.S.; Wang, H.Y.; Shee, D.Y. Measuring e-learning systems success in an organizational context: Scale development and validation. Comput. Hum. Behav. 2007, 23, 1792–1808. [Google Scholar] [CrossRef]
  40. Wu, L.; Chiu, M.L.; Chen, K.W. Defining the determinants of online impulse buying through a shopping process of integrating perceived risk, expectation-confirmation model, and flow theory issues. Int. J. Inf. Manag. 2020, 52, 102099. [Google Scholar] [CrossRef]
  41. Zheng, Y.; Zhao, K.; Stylianou, A. The impacts of information quality and system quality on users’ continuance intention in information-exchange virtual communities: An empirical investigation. Decis. Support Syst. 2013, 56, 513–524. [Google Scholar] [CrossRef]
  42. Wu, J.H.; Wang, Y.M. Measuring KMS success: A respecification of the DeLone and McLean’s model. Inf. Manag. 2006, 43, 728–739. [Google Scholar] [CrossRef]
  43. Hsu, C.L.; Lin, J.C.C. What drives purchase intention for paid mobile apps?–An expectation confirmation model with perceived value. Electron. Commer. Res. Appl. 2015, 14, 46–57. [Google Scholar] [CrossRef]
  44. Leung, L.; Chen, C. E-health/m-health adoption and lifestyle improvements: Exploring the roles of technology readiness, the expectation-confirmation model, and health-related information activities. Telecomm. Policy 2019, 43, 563–575. [Google Scholar] [CrossRef] [Green Version]
  45. Gupta, A.; Yousaf, A.; Mishra, A. How pre-adoption expectancies shape post-adoption continuance intentions: An extended expectation-confirmation model. Int. J. Inf. Manag. 2020, 52, 102094. [Google Scholar] [CrossRef]
  46. Bhattacherjee, A. Understanding information systems continuance: An expectation-confirmation model. MIS Q. 2001, 25, 351–370. [Google Scholar] [CrossRef]
  47. Herodotou, C.; Mystakidis, S. Addressing the retention gap in MOOCs: Towards a motivational framework for MOOCs instructional design. In Proceedings of the 6th Biennial EARLI Conference for Research on Learning and Instruction, Limassol, Cyprus, 25–29 August 2015. [Google Scholar]
  48. Hone, K.S.; El Said, G.R. Exploring the factors affecting MOOC retention: A survey study. Comput. Educ. 2016, 98, 157–168. [Google Scholar] [CrossRef] [Green Version]
  49. An, Y.; Zhu, M.; Bonk, C.J.; Lin, L. Exploring instructors’ perspectives, practices, and perceived support needs and barriers related to the gamification of MOOCs. J. Comput. High Educ. 2020, 33, 1–21. [Google Scholar] [CrossRef]
  50. Khalil, M.; Ebner, M. Clustering patterns of engagement in Massive Open Online Courses (MOOCs): The use of learning analytics to reveal student categories. J. Comput. High Educ. 2017, 29, 114–132. [Google Scholar] [CrossRef]
  51. Saputro, R.E.; Salam, S.; Zakaria, M.H.; Anwar, T. A gamification framework to enhance students’ intrinsic motivation on MOOC. Telkomnika 2019, 17, 170–178. [Google Scholar] [CrossRef] [Green Version]
  52. Hassanzadeh, A.; Kanaani, F.; Elahi, S. A model for measuring e-learning systems success in universities. Expert Syst. Appl. 2012, 39, 10959–10966. [Google Scholar] [CrossRef]
  53. Marjanovic, U.; Delić, M.; Lalic, B. Developing a model to assess the success of e-learning systems: Evidence from a manufacturing company in transitional economy. Inf. Syst. eBus. Manag. 2016, 14, 253–272. [Google Scholar] [CrossRef]
  54. Lin, H.F. Measuring online learning systems success: Applying the updated DeLone and McLean model. Cyberpsychol. Behav. 2007, 10, 817–820. [Google Scholar] [CrossRef]
  55. Bhattacherjee, A.; Lin, C.P. A unified model of IT continuance: Three complementary perspectives and crossover effects. Eur. J. Inf. Syst. 2015, 24, 364–373. [Google Scholar] [CrossRef]
  56. Roca, J.C.; Chiu, C.M.; Martínez, F.J. Understanding e-learning continuance intention: An extension of the Technology Acceptance Model. Int. J. Hum. Comput. Stud. 2006, 64, 683–696. [Google Scholar] [CrossRef] [Green Version]
  57. Chung, N.; Lee, H.; Lee, S.J.; Koo, C. The influence of tourism website on tourists’ behavior to determine destination selection: A case study of creative economy in Korea. Technol. Forecast. Soc. Chang. 2015, 96, 130–143. [Google Scholar] [CrossRef]
  58. Shin, D.H.; Shin, Y.J.; Choo, H.; Beom, K. Smartphones as smart pedagogical tools: Implications for smartphones as u-learning devices. Comput. Hum. Behav. 2011, 27, 2207–2214. [Google Scholar] [CrossRef]
  59. Jumaan, I.A.M.; Hashim, N.H.; Al-Ghazali, B.M. The role of cognitive absorption in predicting mobile internet users’ continuance intention: An extension of the expectation-confirmation model. Technol. Soc. 2020, 63, 101355. [Google Scholar] [CrossRef]
  60. Wang, C.; Teo, T.S.; Liu, L. Perceived value and continuance intention in mobile government service in China. Telemat. Inform. 2020, 48, 101348. [Google Scholar] [CrossRef]
  61. Pozón-López, I.; Higueras-Castillo, E.; Muñoz-Leiva, F.; Liébana-Cabanillas, F.J. Perceived user satisfaction and intention to use massive open online courses (MOOCs). J. Comput. High Educ. 2020, 33, 1–36. [Google Scholar] [CrossRef]
  62. Nasirzadeh, E.; Fathian, M. Investigating the Effect of Gamification Elements on Bank Customers to Personalize Gamified Systems. Int. J. Hum. Comput. Stud. 2020, 144, 102469. [Google Scholar] [CrossRef]
  63. Putz, L.M.; Hofbauer, F.; Treiblmaier, H. Can gamification help to improve education? Findings from a longitudinal study. Comput. Hum. Behav. 2020, 110, 106392. [Google Scholar] [CrossRef]
  64. Ajzen, I. The theory of planned behavior. Organ. Behav. Hum. Decis. Process. 1991, 50, 179–211. [Google Scholar] [CrossRef]
  65. Joo, Y.J.; Park, S.; Shin, E.K. Students’ expectation, satisfaction, and continuance intention to use digital textbooks. Comput. Hum. Behav. 2017, 69, 83–90. [Google Scholar] [CrossRef]
  66. Li, C.Y.; Fang, Y.H. Predicting continuance intention toward mobile branded apps through satisfaction and attachment. Telemat. Inform. 2019, 43, 101248. [Google Scholar] [CrossRef]
  67. Bölen, M.C.; Özen, Ü. Understanding the factors affecting consumers’ continuance intention in mobile shopping: The case of private shopping clubs. Int. J. Mob. Commun. 2020, 18, 101–129. [Google Scholar] [CrossRef]
  68. Hassan, L.; Dias, A.; Hamari, J. How motivational feedback increases user’s benefits and continued use: A study on gamification, quantified-self and social networking. Int. J. Inf. Manag. 2019, 46, 151–162. [Google Scholar] [CrossRef]
  69. Jun, F.; Jiao, J.; Lin, P. Influence of virtual CSR gamification design elements on customers’ continuance intention of participating in social value co-creation: The mediation effect of psychological benefit. Asia Pac. J. Mark. Logist. 2020, 32, 1305–1326. [Google Scholar] [CrossRef]
  70. Tam, C.; Oliveira, T. Understanding the impact of m-banking on individual performance: DeLone & McLean and TTF perspective. Comput. Hum. Behav. 2016, 61, 233–244. [Google Scholar]
  71. Mohammadyari, S.; Singh, H. Understanding the effect of e-learning on individual performance: The role of digital literacy. Comput. Educ. 2015, 82, 11–25. [Google Scholar] [CrossRef]
  72. Gerdenitsch, C.; Sellitsch, D.; Besser, M.; Burger, S.; Stegmann, C.; Tscheligi, M.; Kriglstein, S. Work gamification: Effects on enjoyment, productivity and the role of leadership. Electron. Commer. Res. Appl. 2020, 43, 100994. [Google Scholar] [CrossRef]
  73. Feng, W.; Tu, R.; Hsieh, P. Can gamification increase consumers’ engagement in fitness apps? The moderating role of commensurability of the game elements. J. Retail. Consum. Serv. 2020, 57, 1–9. [Google Scholar] [CrossRef]
  74. Groening, C.; Binnewies, C. “Achievement unlocked!”—The impact of digital achievements as a gamification element on motivation and performance. Comput. Hum. Behav. 2019, 97, 151–166. [Google Scholar] [CrossRef]
  75. Legaki, N.Z.; Xi, N.; Hamari, J.; Karpouzis, K.; Assimakopoulos, V. The effect of challenge-based gamification on learning: An experiment in the context of statistics education. Int. J. Hum. Comput. Stud. 2020, 144, 102496. [Google Scholar] [CrossRef]
  76. Tobon, S.; Ruiz-Alba, J.L.; García-Madariaga, J. Gamification and online consumer decisions: Is the game over? Decis. Support Syst. 2020, 128, 113167. [Google Scholar] [CrossRef]
  77. Rodrigues, L.F.; Oliveira, A.; Costa, C.J. Does ease-of-use contributes to the perception of enjoyment? A case of gamification in e-banking. Comput. Hum. Behav. 2016, 61, 114–126. [Google Scholar] [CrossRef]
  78. Zhang, Q.; Peck, K.L.; Hristova, A.; Jablokow, K.W.; Hoffman, V.; Park, E.; Bayeck, R.Y. Exploring the communication preferences of MOOC learners and the value of preference-based groups: Is grouping enough? Educ. Technol. Res. Dev. 2016, 64, 809–837. [Google Scholar] [CrossRef]
  79. Podsakoff, P.M.; Organ, D.W. Self-Reports in Organizational Research: Problems and Prospects. J. Manag. 1986, 12, 531–544. [Google Scholar] [CrossRef]
  80. Podsakoff, P.M.; MacKenzie, S.B.; Lee, J.-Y.; Podsakoff, N.P. Common method biases in behavioral research: A critical review of the literature and recommended remedies. J. Appl. Psychol. 2003, 88, 879–903. [Google Scholar] [CrossRef] [PubMed]
  81. Pursel, B.K.; Zhang, L.; Jablokow, K.W.; Choi, G.W.; Velegol, D. Understanding MOOC students: Motivations and behaviours indicative of MOOC completion. J. Comput. Assist. Learn. 2016, 32, 202–217. [Google Scholar] [CrossRef]
  82. Li, H.; Liu, Y.; Xu, X.; Heikkilä, J.; van der Heijden, H. Modeling hedonic is continuance through the uses and gratifications theory: An empirical study in online games. Comput. Hum. Behav. 2015, 48, 261–272. [Google Scholar] [CrossRef]
  83. Rauniar, R.; Rawski, G.; Yang, J.; Johnson, B. Technology acceptance model (TAM) and social media usage: An empirical study on Facebook. J. Enterp. Inf. Manag. 2014, 27, 6–30. [Google Scholar] [CrossRef]
  84. Lin, W.S. Perceived fit and satisfaction on web learning performance: IS continuance intention and task-technology fit perspectives. Int. J. Hum. Comput. Stud. 2012, 70, 498–507. [Google Scholar] [CrossRef]
Sustainability 13 08089 g001 550
Figure 1. Processes for managing one course. Source: https://www.icourse163.org/, accessed on 21 June 2021.
Figure 1. Processes for managing one course. Source: https://www.icourse163.org/, accessed on 21 June 2021.
Sustainability 13 08089 g001
Sustainability 13 08089 g002 550
Figure 2. Student’s profile sample. Source: https://www.icourse163.org/, accessed on 21 June 2021.
Figure 2. Student’s profile sample. Source: https://www.icourse163.org/, accessed on 21 June 2021.
Sustainability 13 08089 g002
Sustainability 13 08089 g003 550
Figure 3. Design and structure of a sample course. Source: https://www.icourse163.org/, accessed on 21 June 2021.
Figure 3. Design and structure of a sample course. Source: https://www.icourse163.org/, accessed on 21 June 2021.
Sustainability 13 08089 g003
Sustainability 13 08089 g004 550
Figure 4. Lucky Card game of a MOOC.
Figure 4. Lucky Card game of a MOOC.
Sustainability 13 08089 g004
Sustainability 13 08089 g005 550
Figure 5. The badge provided by a MOOC. Source: https://www.icourse163.org/, accessed on 21 June 2021.
Figure 5. The badge provided by a MOOC. Source: https://www.icourse163.org/, accessed on 21 June 2021.
Sustainability 13 08089 g005
Sustainability 13 08089 g006 550
Figure 6. Proposed research model.
Figure 6. Proposed research model.
Sustainability 13 08089 g006
Sustainability 13 08089 g007 550
Figure 7. Path coefficients of the research model. NOTE: ** p < 0.01; *** p < 0.001.
Figure 7. Path coefficients of the research model. NOTE: ** p < 0.01; *** p < 0.001.
Sustainability 13 08089 g007
Table
Table 1. Demographic distribution.
Table 1. Demographic distribution.
VariablesClassificationNumberPercentage
GenderMale20935.7
Female37764.3
Education backgroundUndergraduate48282.3
Graduate school (M.A.)203.4
Graduate school
(Ph.D. and above)		8414.3
ExperienceLess than 1 month13623.2
1–3 months12721.7
3–6 months11720
6–9 months6410.9
9–12 months356
12 months and above10718.3
Average using time/week1–4 h26044.4
5–9 h15125.8
10–14 h8714.8
15 h and above8815
ResidencyWestern16027.3
Central21536.7
Eastern21136
Table
Table 2. Factor loading, CR, AVE, and Cronbach’s α values.
Table 2. Factor loading, CR, AVE, and Cronbach’s α values.
ConstructsFirst-Order ConstructsItemItem LoadingCRAVECronbach’s α
MOOC platform quality confirmationN/AMCON10.8140.8380.6320.837
MCON20.777
MCON30.794
Information quality N/AIQ10.7710.8390.5650.838
IQ20.731
IQ30.725
IQ40.779
Service qualityN/ASEQ10.7840.8590.6050.858
SEQ20.763
SEQ30.75
SEQ40.811
System qualityN/ASQ10.8180.8620.610.861
SQ20.767
SQ30.752
SQ40.784
MOOC usefulnessN/AMUSE10.8450.8540.6620.853
MUSE20.768
MUSE30.825
MOOC satisfactionN/AMSAT10.7990.8130.5920.809
MSAT20.7
MSAT30.805
Gamification
perceptions		Social interactionSOC10.8340.8810.650.88
SOC20.754
SOC30.794
SOC40.839
ChallengeCHA10.8260.8310.6230.827
CHA20.709
CHA30.827
EntertainmentENT10.8140.850.5870.849
ENT20.722
ENT30.733
ENT40.792
MOOC continued usage intentionN/AMCI10.7950.8330.6240.831
MCI20.756
MCI30.818
Course performanceN/ACP10.7810.8410.570.841
CP20.739
CP30.739
CP40.761
Table
Table 3. Discriminant validity test.
Table 3. Discriminant validity test.
IQSQSEQMCONMUSEGAMMSATMCICP
IQ0.752
SQ0.433 **0.781
SEQ0.457 **0.418 **0.778
MCON0.392 **0.338 **0.366 **0.795
MUSE0.318 **0.288 **0.263 **0.394 **0.814
GAM0.366 **0.312 **0.323 **0.328 **0.178 **0.767
MSAT0.353 **0.307 **0.325 **0.532 **0.514 **0.284 **0.769
MCI0.356 **0.352 **0.364 **0.405 **0.435 **0.425 **0.501 **0.790
CP0.403 **0.361 **0.414 **0.393 **0.454 **0.506 **0.414 **0.575 **0.755
NOTE: ** p < 0.01. MCON = MOOC platform quality confirmation; IQ = Information quality; SQ = System quality; SEQ = Service quality; MUSE = MOOC usefulness; MSAT = MOOC satisfaction; GAM = Gamification; SOC = Social interaction; CHA = Challenge; ENT = Entertainment; MCI = MOOC continued usage intention; CP = Course performance.
Table
Table 4. Common method bias test.
Table 4. Common method bias test.
ComponentInitial Eigen Values
1Total% of VarianceCumulative %
11.59127.71927.719
Extraction Sums of Squared Loadings
Total% of VarianceCumulative %
10.88127.89927.899
Table
Table 5. Measurement model fit.
Table 5. Measurement model fit.
Fit Indicesχ2/dfGFIAGFINFICFIRMRRMSEA
Recommended Value<3.0>0.9>0.8>0.9>0.9<0.08<0.08
Value indices1.5060.9200.9090.9180.9710.0690.029
Table
Table 6. Hypothesis test results.
Table 6. Hypothesis test results.
HypothesisPathβp-ValueR2Remarks
H1IQ→MCON0.312***0.338Supported
H2SQ→MCON0.1720.001Supported
H3SEQ→MCON0.217***Supported
H4MCON→MUSE0.489***0.239Supported
H5MCON→GAM0.463***0.215Supported
H6MCON→MSAT0.453***0.553Supported
H7MUSE→MSAT0.375***Supported
H8GAM→MSAT0.0620.189Not supported
H9MUSE→MCI0.23***0.518Supported
H10GAM→MCI0.365***Supported
H11MSAT→MCI0.335***Supported
H12MUSE→CP0.275***0.621Supported
H13GAM→CP0.392***Supported
H14MCI→CP0.331***Supported
H15GAM→SOC0.675***0.675Supported
→CHA0.553***0.533
→ENT0.533***0.553
NOTE: *** p < 0.001.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Yang, Q.; Lee, Y.-C. The Critical Factors of Student Performance in MOOCs for Sustainable Education: A Case of Chinese Universities. Sustainability 2021, 13, 8089. https://doi.org/10.3390/su13148089

AMA Style

Yang Q, Lee Y-C. The Critical Factors of Student Performance in MOOCs for Sustainable Education: A Case of Chinese Universities. Sustainability. 2021; 13(14):8089. https://doi.org/10.3390/su13148089

Chicago/Turabian Style

Yang, Qin, and Young-Chan Lee. 2021. "The Critical Factors of Student Performance in MOOCs for Sustainable Education: A Case of Chinese Universities" Sustainability 13, no. 14: 8089. https://doi.org/10.3390/su13148089

APA Style

Yang, Q., & Lee, Y. -C. (2021). The Critical Factors of Student Performance in MOOCs for Sustainable Education: A Case of Chinese Universities. Sustainability, 13(14), 8089. https://doi.org/10.3390/su13148089

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop