Comparative Study on Barriers of Supply Chain Management MOOCs in China: Online Review Analysis with a Novel TOPSIS-CoCoSo Approach

Abstract

To enhance the effectiveness of supply chain talent education, higher education institutions and other organisations have started to develop and use Massive Open Online Courses (MOOCs) in their training programs. However, the problem is that the design and delivery of supply chain management MOOCs can be inappropriately presented and, thus, ineffective, especially for educational teams with fewer teaching experiences of MOOCs. This eventually makes it hard for the students’ learning outcomes to meet the industrial requirements of supply chain experts. Motivated by such a problem, this paper aims to improve the design and delivery of supply chain management MOOCs to enhance student learning outcomes. To achieve this goal, the research method adopted in this paper is to analyse online reviews in a widely-used Chinese MOOC platform with a novel TOPSIS-CoCoSo approach, aiming to identify the barriers to supply chain management MOOCs and their potential solutions. The results of this study show that 16 barriers to MOOCs are identified from the online reviews and then ranked based on their severity of reducing learning outcomes. The perceptions of the severity of the barriers to students and lecturers are compared, and the solutions to the barriers are then discussed. In addition, our comparison indicates that although students and lecturers have similar perceptions of severity for the majority of the barriers, they have significant disagreements on certain barriers. The significance of this study is that it can inform lecturers in supply chain management or relevant disciplines to better design and deliver their MOOC content, as well as contribute to the existing literature by providing new methodological tools for educational analysis. Also, this study highlights the necessity of comparative study in the MOOC online review analysis.

1. Introduction

Nowadays, supply chain management has become vitally important in industry. Effective supply chain management can coordinate the multiple companies in the market and enable better collaboration among them [1]. In China, the importance of supply chains has not only been recognised by companies but also by the government. In recent years, multiple policies have been developed to stimulate the good development and management of supply chains [2]. However, good development of supply chains requires sufficient talents who have rich theoretical and practical supply chain management knowledge. China still faces a shortage of supply chain talent, and the industrial need for good supply chain management is hard to cover. Therefore, new and effective education approaches should be built to fulfil the needs of supply chain management talent.

The development of digital and online technologies has made Massive Open Online Courses (MOOCs) a popular method for students to study specialised knowledge [3]. The emergence of Coursera, edX, icourse163, and many other MOOC platforms enables students to learn courses developed by top universities and professional experts easily, and such an advantage becomes more significant during the COVID-19 pandemic. In the supply chain management discipline, MOOCs are effective in educating talent. In recent years, universities in China have encouraged their faculties to develop MOOCs for supply chain management or relevant courses. By using MOOCs, students can learn and recap the course content about supply chain management knowledge at their own pace and discuss and work with their classmates online to finish the course project [4].

However, although in China, using MOOCs to learn supply chain management knowledge has gradually become popular, there are still multiple barriers that can reduce the learning outcomes and need to be removed. For example, students cannot easily adapt themselves to the e-learning approach and have little motivation for MOOCs [5], probably due to the tradition of the offline classroom teaching approach. Also, the designers and/or the lecturers of the MOOCs can be new to this teaching mode, with little experience in designing and delivering the MOOC content properly. The existence of various barriers largely hinders the good learning outcomes of students in supply chain management MOOCs.

However, so far, there has been little research focusing on clarifying the barriers to supply chain management MOOCs in China, and the solution to them remains unexplored. Therefore, to fully uncover the barriers of MOOCs in supply chain management learning and develop the solutions, this paper aims to explore the following research question: What are the barriers of supply chain management MOOCs in China, and how can they be solved?

To explore this question, in this paper, we applied a mixed method. We collected the online reviews of supply chain management MOOCs in a widely used Chinese MOOCs platform and developed a novel model integrating Technique for Order Preference by Similarity to Ideal Solutions (TOPSIS) [6] and COmbined COmpromise SOlution (CoCoSo) [7] to analyse them. We identified 16 important barriers that could negatively influence MOOC learning outcomes and comparatively analysed them from students’ and lecturers’ perspectives. Based on our findings, we provided effective educational strategies to support the design and delivery of supply chain management MOOCs.

We believe this paper has the following contributions:

• Our paper identified 16 barriers that negatively influenced the learning outcomes of supply chain management MOOCs in China. Although the previous literature studied the MOOCs’ barriers, there are few studies focusing on Chinese supply chain management courses. As the Chinese market has its own unique features and policies, the supply chain courses can have different focuses than other countries. Our study can inform MOOC designers and instructors of a better course design and delivery, significantly improving the MOOC quality to better fit the local features;
• Our paper provided a comparative study on the perception of barrier severity between students and lecturers. To the best of our knowledge, this paper is the first to adopt this perspective to study supply chain management MOOCs in China. Our results showed that although the perceptions between students and lecturers are generally consistent, there are still some diversities. Our study, thus, can contribute to informing the MOOC designer that course curriculum and content development should be student-centred instead of lecturer-centred so that a good learning outcome can be achieved;
• Finally, our study proposed a novel analytical method by integrating TOPSIS and CoCoSo. This method can lead to a better MOOC design and evaluation. Although TOPSIS is a widely used technique, its value for determining decision makers’ weights is largely under-explored. By extending the two ideal solutions of TOPSIS to three and integrating them with CoCoSo, the objective determination of decision makers’ weights can be well achieved. This enables our method to generate a more robust and effective decision-making than the existing literature. Our paper, thus, contributes to the existing literature on MOOCs and supply chain management education by providing a new and effective analytical tool.

This paper has six sections. After the introduction, the second section reviews the relevant literature of this paper, followed by the methodology proposed in the third section. The fourth section reports the results of MOOC barriers, and their implications and solutions are discussed in the fifth section. Finally, a conclusion is drawn in section six, with research implications, research limitations, and future directions suggested.

2. The Literature Review

This study is concerned with three streams of the literature, namely, factors influencing MOOC learning outcomes, supply chain management education, and MOOC evaluation through online reviews.

2.1. Factors Influencing MOOCs Learning Outcome

The previous literature has intensively examined the factors influencing the learning outcomes of students taking MOOCs. Ref. [8] indicated that the learners’ motivation and participation are tightly linked to the MOOC learning outcomes, but such effects vary between different courses. Ref. [9] found that MOOCs, when combined with other types of educational approaches like flipped classrooms or game-based learning, can improve the learning outcomes of students, especially for those less confident in the course content. Ref. [10] found that different kinds of learner engagements could significantly influence MOOC learning outcomes. Ref. [11] used machine learning algorithms to analyse the learning behaviours of the MOOC learner, and they found that the MOOC video-watching patterns could be a significant predictor for the final learning outcomes. Ref. [12] applied meta-analysis for factors influencing MOOC learning outcomes among different courses and found that there were eight important indicators covering different aspects of learning behaviours and activities. Ref. [13] also indicated that learners’ motivation could influence the learning outcomes, but such an influence could interact with multiple factors, such as course design, interaction with lecturers, and learning strategies. Ref. [14] revealed that MOOCs’ gamification and personalised environments could improve the students’ engagement, leading to a better learning outcome. Ref. [15] conducted a literature review to systematically summarise the factors of students’ dropout rates in MOOCs, and they found that social, psychological, course-related factors, etc., could influence dropout rates, but the effects of different factors could be the opposite. Ref. [16] conducted a meta-analysis to examine the effect of self-regulation on the learning outcomes of MOOCs, and they found that self-regulation was important for good learning outcomes of MOOCs, although the significance of such an effect could be moderated by certain environmental or students’ properties.

Although current studies have investigated multiple influential factors of MOOC learning outcomes, there seems to be a lack of research on MOOCs specialised in the supply chain management discipline, especially from a Chinese educational background. The lack of such research undermines the full understanding of how MOOC learning outcomes can be improved. This is because, compared with other disciplines, supply chain management has its own differences in course curriculums and teaching/learning modes due to its knowledge structures. Also, China has unique market features and policies, which lead to different focuses on MOOC content. Therefore, this paper tends to fill this gap and contributes to the literature by examining the factors influencing Chinese supply chain MOOC learning outcomes using an online review analysis approach.

2.2. Supply Chain Management Education

The importance of supply chain management attracts the increasing attention of industries. Supply chains with high efficiency and flexibility can enable companies to perform better. To achieve this, supply chain talents are indispensable. To cultivate supply chain talents and meet industrial needs, existing publications explored educational topics from multiple perspectives. Ref. [17] investigated sustainable supply chain management education from the perspective of engineering discipline. Focusing on Brazilian higher education institutions, they identified the current status of sustainable supply chain education and the improvement opportunities. Ref. [18], by surveying the previous literature relevant to logistics and supply chain management education, spotted the key competencies, skills, and courses needed for supply chain talents. Ref. [19] studied the game-based learning approach in supply chain management education. Based on the TagScan program, they developed a virtual role-playing game to teach students about information flows in supply chains. Ref. [20] utilised a Tableau dashboard in the supply chain class to create a beer supply chain game to enable students to learn supply chain operations knowledge, as well as to support lecturers in analysing real-time learning data. Ref. [21] applied RFID and barcode technology to virtually simulate a retailing environment in classes to teach students about inventory management, which is an important activity in supply chains. Ref. [22] explored how a lecturer’s communication behaviour can influence the students’ affective learning in online supply chain management courses, through which the effects of the lecturer–student relationship on online learning experiences were studied. Ref. [23] adopted a novel approach to teach supply chain management knowledge. Students were asked to create a work of art for certain supply chain ethical cases, and the authors reflected students can better understand the course content with higher enjoyment. Ref. [24] found that current supply chain talent training may not fulfil industrial needs completely, and they developed a participatory educational framework to enhance the learning outcomes of students.

Compared with the above literature on MOOC education, there are relatively few studies on the supply chain management of MOOCs. Some exceptions are as follows. Ref. [25] conducted an online search study for the sources of supply chain workforce training and confirmed that MOOCs are one of the most accessible sources for companies to train their supply chain experts. Ref. [26] examined the factors influencing dropout rates in the MITx MicroMasters^® Program in Supply Chain Management, which is a famous MOOC program in the supply chain education field. The intervention was then developed for dropout reduction after multiple analyses, but no significant reduction in MOOC dropout rate was found after the intervention. Following the findings of [26], ref. [27] developed four different interventions and conducted A/B tests and natural experiments to examine their effectiveness. The authors found that course content modification can be a useful intervention to reduce dropout rates.

Although studies have started to focus on several facets of supply chain management MOOCs, thorough explorations of the barriers to good learning outcomes are largely lacking. This would directly hinder ways of improving the quality of supply chain education in the current digital age, as well as reduce the effectiveness of utilising valuable MOOC resources. Therefore, this study tends to explore the barriers to supply chain management MOOC learning outcomes, paving the way for improving educational effectiveness in supply chain fields.

2.3. MOOC Online Review Analysis

As a feedback approach, online reviews have been widely adopted in MOOC evaluation to improve teaching quality. Through opinions collected in online reviews, learners can better understand the course content and quality, while the lecturers can update their content and teaching approaches. The existing literature has explored MOOC online reviews for different purposes. For example, Ref. [28] explored how online review properties could influence the MOOC registration and completion ratio, and they found that such an influence was multi-faceted and could bring diverse effects to learners depending on their features. Ref. [29] developed a curriculum evaluation system based on MOOC reviews using the Latent Dirichlet Allocation and text classification model, by which they identified five key factors that should be focused on curriculum evaluation. Ref. [30] examined the effects of replying to online reviews on MOOC learning satisfaction. They found that replying behaviours could probably enhance the learners’ MOOC satisfaction and learning perceptions. Ref. [31] analysed the online reviews of top-rated Coursera MOOCs with exploratory analysis, sentiment modelling, and correlation analysis. They found that course quality could be enhanced by design and material quality or lecturer demonstration quality, depending on the types of courses. Ref. [32] used big data technologies to analyse online review texts and identified important factors for MOOC learning experiences. They categorised the factors into MOOC delivery factors and subject matter factors, informing the design of MOOCs. Ref. [33] studied the MOOC learner behaviours through online review analysis. They identified that learners who completed the course had distinct behaviours compared with those without completion and also identified the learners’ behaviour dynamics over time. Ref. [34] applied Latent Dirichlet Allocation and fuzzy-set qualitative comparative analysis to computer science MOOC reviews, and they found the factor configurations for high and low MOOC satisfactions. Ref. [35] proposed an analytic framework for automatically analysing MOOC online reviews using a deep neural network. Such a framework can facilitate a rapid understanding of the learner experiences without intensively qualitative review coding.

Recently, multi-criteria decision-making (MCDM) approaches like the Analytic Hierarchy Process (AHP) have been introduced into MOOC review analysis to support course quality evaluation. For example, Ref. [36] combined the text mining algorithms with AHP to develop a diagnostic tool for MOOC quality evaluation. However, it can be observed that although MCDM approaches are promising to capture useful course feedback information in online reviews, such a direction is still in its infancy. Therefore, this study tends to fill this gap and develop an effective method to utilise MCDM to support MOOC online review analysis.

2.4. Research Gap Summary

The above literature review reveals that although there are multiple studies focusing on factors of MOOCs learning outcomes, supply chain management education, and MOOCs evaluation through online reviews alone, few explorations have been completed to integrate them and analyse online reviews of supply chain management MOOCs to investigate the barriers to effective course learning and identify their solutions. Therefore, this study aims to fix this research gap by analysing Chinese supply chain management MOOC barriers from course reviews using a novel TOPSIS-CoCoSo model through a comparative investigation between students and lecturers.

3. Methodology

In this section, we reported our methodology for studying the barriers to Chinese supply chain management MOOCs through a comparative study in online reviews. Figure 1 presents our methodological flow.

3.1. Online Review Collection and Coding

In this paper, we studied the barriers to supply chain management MOOC learning outcomes from an online review analysis perspective. To fully capture the barriers reflected by learners, we collected our online review data from one of the most widely used MOOC platforms in China, called “Chinese University MOOC” (icourse163.org, accessed on 2 November 2023). The online reviews of all MOOCs with their titles containing “supply chain” were collected by using Bazhuayu version 8.6.4 (a web crawler software). Such a process led to 15 MOOCs being identified, and the statistics of their reviews are summarised in

Table 1. Statistics of supply chain management MOOC reviews.

University Type	Number of MOOCs	Number of Online Reviews
Double First-Class Initiative University (Top university)	6	550
University	5	390
College	4	643
Total	15	1583

. It can be observed that the institutions offering these MOOCs range from top universities to colleges, with a total number of reviews equal to 1583. The ending date for collecting reviews is 2 November 2023. The time span of the collected online reviews ranges from 27 February 2019 to 29 October 2023, meaning that our dataset covers almost five years of reviews. As the timespan of our dataset is sufficiently long, and the online reviews contain all supply chain titled MOOCs in the platform and cover different levels of universities/colleges, it can be reasonably stated that our dataset is representative and can effectively and inclusively reveal the learners’ evaluation on the supply chain management MOOCs in China. By analysing the dataset, the barriers to supply chain management MOOCs reducing the students’ learning outcomes in China can be effectively identified.

After collecting online reviews, the first author carefully checked the review content and found that although the majority of the reviews were positive, there were 76 negative reviews, which could reflect the dissatisfaction of the learners. All negative reviews were manually identified by the first author and then read by the other two authors. To spot the barriers from the negative reviews, the authors followed the instructions of content analysis in the previous literature (e.g., [37,38]). Specifically, the following procedures were applied. First, the three authors read the 76 negative reviews to familiarise themselves with the learners’ opinions. After that, the three authors independently coded each of the reviews and generated 19 initial codes to describe the negative reflections of learners. Based on the initial codes, three authors then independently merged the codes into the categories describing the barriers to good learning outcomes of supply chain management MOOCs. This procedure led to 16 barriers in total, but only 12 barriers achieved agreement among three authors, with inconsistent thoughts existing in the rest of the 4 barriers. To achieve the agreement, the three authors discussed together twice, based on the findings of the previous literature as well as their own teaching experience in supply chain management. After the discussion, the 4 barriers were modified and finalised, with the agreement achieved among all authors. This eventually led to the final number of barriers to 16. As our coding procedures followed the systematic approaches and the agreements among authors were fully achieved in the end, it can be stated that the coding reliability among authors is acceptable, and the barriers identified are representative to be used for the following TOPSIS-CoCoSo model.

3.2. Extend TOPSIS for Decision Maker Weights

After identifying the barriers from online reviews, it is necessary to find and rank the barriers by their severity. This is because the barriers can post different levels of negative influence on the learning outcomes, and those with the highest negative influences should be addressed first. To do so, we followed the previous literature and applied an MCDM procedure (e.g., [39,40,41,42,43]). We invited decision makers (DMs) to rate the severity of each barrier to their learning outcomes and aggregate each DM’s rating to calculate the final severity level of each barrier. In this study, the DMs we invited were 11 lecturers and 44 students who were familiar with e-learning and supply chain management discipline, totalling 55 participants in our study.

To compare the perception difference between lecturers and students, we separately asked the lecturers and students to rate the barrier’s severity. The reasons why we invited more students than lecturers are twofold. First, as MOOC online reviews are posted by learners rather than instructors, the thoughts and ideas reflected from the reviews are essentially adopting a student perspective. Therefore, inviting more students can better capture the essence of the barriers reflected in the online reviews. Second and more importantly, this paper adopts the student-centred principle [44,45], advocating that the MOOCs’ design should fulfil students’ needs rather than meet lecturers’ perceptions. Therefore, the lecturers’ opinions in our study essentially work as a complementary role to the students, leading to more students being invited than lecturers. We circulated a questionnaire to collect the DMs’ rates. Specifically, for each barrier, we asked the students the following question, “Based on your understanding, if you use MOOCs to learn supply chain management, how significantly do you think this barrier negatively influences your learning outcome?” while asking the lecturers the following question, “Based on your understanding, if your students use MOOCs to learn supply chain management, how significantly do you think this barrier negatively influence your students’ learning outcome?”. We ask the DMs to rate the severity of the barriers between 0 and 9, where 0 means no negative influence, and 9 represents an absolutely significant negative influence. The higher the rate, the higher the severity of the barriers. After collecting all rates from 55 DMs, we found there were 3 questionnaires that chose “option 0” for all barriers, “option 9” for all barriers, and 15 consecutive chose “option 0”, respectively. Therefore, we discarded these three questionnaires, leading to the final sample containing rates from 42 students and 10 lecturers. Compared with the previous literature [46,47,48], our sample size is significantly larger and, thus, can effectively capture the perceptions of the barriers of supply chain management MOOCs. Therefore, it can be stated that our results are of acceptable reliability and robustness, with potential bias largely eliminated. Based on the collected results, we then separately calculated lecturers’ and students’ rating results.

However, when calculating the DM’s rating results, the previous literature rarely considered the difference between DMs and aggregated rating results by treating each DM equally. Although such a way of aggregation is easy to implement, it fails to consider the DMs’ different decision levels [39,49], which can be caused by their unique backgrounds or knowledge. This can probably lead to a biased result. To better deal with this problem, this study aims to consider the differences in all DM’s decision processes and reflect them by giving different weights for DMs.

To realise it, we modified the extended TOPSIS weighting approach in [39,49] in our study and developed a new MCDM process. Specifically, we assume that there are $m$ different barriers, with their severity rated by $n$ DMs. Therefore, we have the following rating matrix $\mathit{x}$, with its element as $x_{ij}$ representing $j^{th}$ DM’s rating for $i^{th}$ barrier:

$$\mathit{x}=\left[\begin{array}{cccc}{x}_{11}& x_{12}& \dots & x_{1n}\\ x_{21}& x_{22}& \dots & x_{2n}\\ \dots & \dots & \dots & \dots \\ x_{m1}& x_{m2}& \dots & x_{mn}\end{array}\right];i=1,2,\dots ,m;j=1,2,\dots ,n.$$

(1)

To conduct MCDM, following the previous literature (Yazdani et al., 2019 [7]), we normalise $\mathit{x}$ to

$$\mathit{r}=\left[\begin{array}{cccc}{r}_{11}& r_{12}& \dots & r_{1n}\\ r_{21}& r_{22}& \dots & r_{2n}\\ \dots & \dots & \dots & \dots \\ r_{m1}& r_{m2}& \dots & r_{mn}\end{array}\right];i=1,2,\dots ,m;j=1,2,\dots ,n.$$

(2)

$r_{ij}$ is equal to

$$r_{ij}=\frac{x_{ij}-\underset{i}{min}{x}_{ij}}{\underset{i}{max}{x}_{ij}-\underset{i}{min}{x}_{ij}};j=1,2,\dots ,n$$

(3)

After normalisation, we calculated the following three $m\times 1$ matrixes, namely, $\overline{\mathit{D}}$, ${\mathit{D}}^{\mathit{l}}$, and ${\mathit{D}}^{\mathit{e}}$:

$$\overline{\mathit{D}}=({\overline{r}}_i{)}_{m\times 1};{\overline{r}}_i=\frac{1}{n}{\sum }_{j=1}^n r_{ij};i=1,2,\dots ,m$$

(4)

$${\mathit{D}}^{\mathit{l}}=(r_i^l{)}_{m\times 1};r_i^l=\underset{j}{min}{r}_{ij};i=1,2,\dots ,m$$

(5)

$${\mathit{D}}^{\mathit{e}}=(r_i^e{)}_{m\times 1};r_i^e=\underset{j}{max}{r}_{ij};i=1,2,\dots ,m$$

(6)

After that, based on [39,50], we derived ${\mathit{S}}^{+}$, ${\mathit{S}}^{\mathit{l}-}$, and ${\mathit{S}}^{\mathit{r}-}$ as follows. ${\mathit{S}}^{+}$ is the positive ideal solution (PIS) matrix, while ${\mathit{S}}^{\mathit{l}-}$ and ${\mathit{S}}^{\mathit{r}-}$ are the left and right negative ideal solution (NIS) matrixes, respectively.

$${\mathit{S}}^{+}={\left(s_j^{+}\right)}_{1\times n};s_j^{+}={\left({\sum }_{i=1}^m (r_{ij}-{\overline{r}}_i{)}^2\right)}^{1/2};j=1,2,\dots ,n.$$

(7)

$${\mathit{S}}^{\mathit{l}-}={\left(s_j^{l-}\right)}_{1\times n};s_j^{l-}={\left({\sum }_{i=1}^m (r_{ij}-r_i^l{)}^2\right)}^{1/2};j=1,2,\dots ,n.$$

(8)

$${\mathit{S}}^{\mathit{r}-}={\left(s_j^{r-}\right)}_{1\times n};s_j^{r-}={\left({\sum }_{i=1}^m (r_{ij}-r_i^e{)}^2\right)}^{1/2};j=1,2,\dots ,n.$$

(9)

As argued by [39,49,50], the closer a decision is to the average rating, the higher a DM’s decision level is; the farther a decision is away from the average rating, the lower a DM’s decision level is. Therefore, the DMs with higher decision levels should be given higher weights. Based on the principle of TOPSIS, the following formulae are applied to calculate each DM’s weight [39]:

$$c{c}_j=\frac{s_j^{l-}+s_j^{r-}}{s_j^{+}+s_j^{l-}+s_j^{r-}};j=1,2,\dots ,n.$$

(10)

$$w_j=\frac{c{c}_j}{{\sum }_{j=1}^n c{c}_j};j=1,2,\dots ,n.$$

(11)

3.3. CoCoSo Analysis

After calculating the weights of DMs, CoCoSo was applied to rank the barriers based on their severity in negatively influencing supply chain management MOOC learning outcomes. The reason why CoCoSo was selected is because of its robustness in the decision process [7]. To conduct CoCoSo analysis, the following procedures were applied.

First, based on the normalised matrix, $\mathit{r}$, the weighted arithmetic mean ($E_i$) and weighted product mean ($P_i$) for each barrier considering the DMs’ weights were calculated as follows:

$$E_i={\sum }_{j=1}^n \left(w_j{r}_{ij}\right)$$

(12)

$$P_i={\sum }_{j=1}^n {\left(r_{ij}\right)}^{w_j}$$

(13)

After that, according to [7,51], three aggregators for $E_i$ and $P_i$ were calculated sequentially:

$$k_{ia}=\frac{P_i+E_i}{{\sum }_{i=1}^m \left(P_i+E_i\right)}$$

(14)

$$k_{ib}=\frac{E_i}{\underset{i}{min}{E}_i}+\frac{P_i}{\underset{i}{min}{P}_i}$$

(15)

$$\begin{array}{c}{k}_{ic}=\frac{\lambda E_i+\left(1-\lambda \right)P_i}{\left(\lambda \underset{i}{max}{E}_i+\left(1-\lambda \right)\underset{i}{max}{P}_i\right)};0⩽\lambda ⩽1\end{array}$$

(16)

$\lambda$ stands for the reliability of the CoCoSo, and usually, it is set equal to 0.5 [51].

Finally, the final ranks of the barriers were determined by a compound value of $k_{ia}$, $k_{ib}$, and $k_{ic}$ as follows:

$$k_i=(k_{ia}{k}_{ib}{k}_{ic}{)}^{\frac{1}{3}}+\frac{1}{3}(k_{ia}+k_{ib}+k_{ic})$$

(17)

As $k_i$ reflects the severity of the barriers, the larger the $k_i$, the higher the rank of the barrier, meaning that such a barrier will pose a more significantly negative influence on MOOC learning outcomes.

3.4. Implementation of TOPSIS-CoCoSo Method

The above sections introduce the mechanisms of the newly proposed method. Here, the method of implementation is presented. The assumption of this method is that each DM can have different weights based on their decision levels [39,49,50], and the lecturers and learners tend to perceive the barriers from different perspectives. Therefore, the implementation should consider the possible heterogeneities of DMs. The following steps depict the TOPSIS-CoCoSo implementation processes to enhance the reproducibility of our method:

Step 1. Data collection. Based on the barriers identified, the questionnaires are separately circulated to students and lecturers to collect their rates on the severity of barriers to reduce the learning outcomes of supply chain management MOOCs;

Step 2. Data preprocessing. The collected questionnaires are checked, and invalid questionnaires are discarded. After that, the rest of the questionnaires are divided into two groups, namely, student rates and lecturer rates;

Step 3. DMs’ weight calculation. The rates in each group are then separately normalised based on Equation (3), followed by the formation of $\overline{\mathit{D}}$, ${\mathit{D}}^{\mathit{l}}$, and ${\mathit{D}}^{\mathit{e}}$ in each group. After that, the ${\mathit{S}}^{+}$, ${\mathit{S}}^{\mathit{l}-}$, and ${\mathit{S}}^{\mathit{r}-}$ in each group are calculated based on Equations (7)–(9). Finally, the weights of each student and lecturer are calculated based on Equations (10) and (11);

Step 4. Criterion ranking. Based on the DMs’ weights in each group, the CoCoSo values for barriers in each group are separately calculated using Equations (12)–(17). After that, the barriers are ranked in each group based on their CoCoSo values in that group;

Step 5. Rank comparison. Due to the different ranks in the two groups, the rank difference is calculated (see Section 4 and Figure 2 for details) to compare the perceptions between students and lecturers.

In the next section, the detailed numerical results are presented to illustrate the implementations of the proposed TOPSIS-CoCoSo method.

4. Results

Based on the proposed MCDM methodology, the results were derived and reported in this section.

First, based on the online review collection and coding, 16 barriers were finally identified.

Table 2. Barriers and explanations.

Barrier No.	Explanation
B1	The MOOC content is not tightly linked to the supply chain practices and is lacking in explanation and analysis for real supply chain management cases.
B2	The MOOC content is out of date and does not cover the cutting-edge knowledge in supply chain management;
B3	The MOOC content is too easy or too difficult for learners;
B4	The MOOCs do not highlight and explain the key or difficult points clearly;
B5	There are mistakes in the course content of supply chain management;
B6	The MOOCs are not attractive, and the lecturer keeps reading slides;
B7	The MOOC learning resources are not enough to fulfil the learning needs, such as lacking in reference books, exercise, or reading materials;
B8	The MOOC videos are of low quality and/or not appropriate for teaching purposes;
B9	The class pace is too fast to give enough time for learners to think and digest the taught knowledge;
B10	The MOOCs lack sufficient course discussion on the forum;
B11	The lecturer speaks too fast or slow or has a strong accent, causing problems in understanding the course content;
B12	The lecturer’s body language and appearance are not appropriate enough for teaching;
B13	The after-class exercise is not properly designed. It is too easy/difficult or too much/little for learners;
B14	The evaluation of the learning performance is not properly designed, lacking in clear evaluation criteria, personalised evaluation approaches, or timely feedback;
B15	The MOOC system is of low performance, with system errors occurring in the process of video playing, learning records, and exercise presentation;
B16	The MOOC system is not user-friendly, and lecturers or students need to spend much time to understand the usage of the system.

lists the barriers and their explanations. The table reveals that the barriers identified from online reviews essentially cover rich facets of supply chain management MOOCs, such as course content, learning materials, MOOC systems, lecturers, and student evaluations, indicating that our online review analysis results are inclusive.

After that, we invited participants to rate the severity of each barrier in negatively influencing MOOC learning outcomes, and the rate statistics of students and lecturers are separately reported in

Table 3. Rating results of students.

Barriers	Max Rating	Min Rating	Average Rating	Rating Standard Deviation
B1	9	0	5.261904762	2.518802695
B2	9	0	4.928571429	2.550876054
B3	9	0	4.095238095	2.206526061
B4	8	0	3.357142857	2.397370801
B5	9	0	3.285714286	3.387824068
B6	9	0	4.404761905	2.499361126
B7	7	0	3.619047619	1.833993124
B8	6	0	2.166666667	1.751886324
B9	8	0	3.142857143	2.42518723
B10	8	0	3.047619048	2.262659572
B11	9	0	2.476190476	2.276476688
B12	7	0	1.642857143	1.736571174
B13	8	0	3.119047619	2.349907948
B14	9	0	3.595238095	2.253014499
B15	9	0	2.976190476	2.493778554
B16	9	0	3.047619048	2.62216054

and

Table 4. Rating results of lecturers.

Barriers	Max Rating	Min Rating	Average Rating	Rating Standard Deviation
B1	9	1	5.6	2.836272985
B2	9	0	5.6	2.716206505
B3	7	0	4.2	2.299758441
B4	8	2	5.2	2.485513584
B5	9	0	2.9	3.247221034
B6	9	1	5.4	2.75680975
B7	7	3	5.1	1.663329993
B8	7	1	3.7	2.002775851
B9	6	1	4.3	1.888562063
B10	9	1	5.2	2.780887149
B11	6	0	2.9	1.852925615
B12	6	0	2.8	2.201009869
B13	8	1	4.3	2.626785107
B14	9	1	4.9	2.960855732
B15	6	0	3.5	2.121320344
B16	7	0	3.5	2.273030283

.

To conduct a comparative study on the perception of barrier severity from both the lecturer’s and student’s perspectives, we separately calculated each DM’s weight using the extended TOPSIS. The results for student weights and lecturer weights are reported in

Table 5. DMs’ weights for students and lecturers.

Students’ weights
No.1	No.2	No.3	No.4	No.5	No.6	No.7	No.8	No.9	No.10
0.024999	0.023954	0.022924	0.025049	0.024232	0.02483	0.024387	0.023638	0.023503	0.023461
No.11	No.12	No.13	No.14	No.15	No.16	No.17	No.18	No.19	No.20
0.024027	0.023298	0.02379	0.024033	0.023898	0.022641	0.024221	0.024254	0.024337	0.023339
No.21	No.22	No.23	No.24	No.25	No.26	No.27	No.28	No.29	No.30
0.023661	0.024272	0.023869	0.023984	0.024317	0.024257	0.023016	0.023814	0.023507	0.021896
No.31	No.32	No.33	No.34	No.35	No.36	No.37	No.38	No.39	No.40
0.024278	0.023949	0.024272	0.022954	0.025267	0.023312	0.023319	0.02245	0.022561	0.023895
No.41	No.42
0.024215	0.024121
Lecturers’ weights
No.1	No.2	No.3	No.4	No.5	No.6	No.7	No.8	No.9	No.10
0.099478	0.100941	0.103616	0.102857	0.098129	0.096711	0.095853	0.103599	0.099438	0.099376

.

Based on the DMs’ weights, the CoCoSo results can be calculated. In

Table 6. CoCoSo results based on students’ ratings.

Barriers	${\mathit{E}}_{\mathit{i}}$	${\mathit{P}}_{\mathit{i}}$	${\mathit{k}}_{\mathit{i}\mathit{a}}$	${\mathit{k}}_{\mathit{i}\mathit{b}}$	${\mathit{k}}_{\mathit{i}\mathit{c}}$	${\mathit{k}}_{\mathit{i}}$	Rank
B1	0.706056813	39.63894508	0.080367412	8.433037104	1	4.049526082	1
B2	0.642216492	38.57095088	0.078112793	7.823697761	0.971946103	3.798524212	2
B3	0.515077113	38.30388395	0.077327532	6.723399702	0.96217522	3.381460938	4
B4	0.401880372	35.21100731	0.070941021	5.56996851	0.882708787	2.878458201	7
B5	0.351576076	20.74465845	0.042023787	4.260453627	0.522895862	2.062526994	13
B6	0.565834057	36.51601218	0.073867192	7.047341732	0.919118714	3.462246235	3
B7	0.481806977	33.50788306	0.067707604	6.147781489	0.842475857	3.057842221	6
B8	0.218639043	25.25797505	0.050749521	3.401715551	0.631468903	1.839017262	15
B9	0.379084827	29.42749331	0.059374828	5.023548614	0.738792334	2.544583301	9
B10	0.342038738	30.34851926	0.061135719	4.763743884	0.760702852	2.466950144	11
B11	0.250024904	25.31978152	0.050935161	3.673086226	0.633778789	1.943879777	14
B12	0.117282807	16.4278247	0.032957923	2	0.410090636	1.114466035	16
B13	0.34951818	32.26604378	0.064970335	4.944240781	0.808416419	2.577203209	8
B14	0.443798981	36.25366137	0.073101494	5.990852083	0.909591241	3.060304643	5
B15	0.350339334	30.28641112	0.061028535	4.830737409	0.759369166	2.490913587	10
B16	0.356321964	27.4544347	0.055399143	4.709358684	0.689323469	2.382482059	12

and

Table 7. CoCoSo results based on lecturers’ ratings.

Barriers	${\mathit{E}}_{\mathit{i}}$	${\mathit{P}}_{\mathit{i}}$	${\mathit{k}}_{\mathit{i}\mathit{a}}$	${\mathit{k}}_{\mathit{i}\mathit{b}}$	${\mathit{k}}_{\mathit{i}\mathit{c}}$	${\mathit{k}}_{\mathit{i}}$	Rank
B1	0.727793351	9.550034968	0.077980451	6.450022758	1	3.30460635	1
B2	0.661150776	8.689927195	0.070948965	5.863350961	0.909830139	3.004730471	4
B3	0.483232116	8.409756854	0.067473329	4.850412306	0.865259537	2.584392951	8
B4	0.652081366	9.484848603	0.07691142	6.033805065	0.986291039	3.136328428	3
B5	0.277753353	4.61426598	0.037116973	2.727685352	0.475977919	1.444162164	15
B6	0.663602703	9.502506777	0.077132812	6.099266128	0.989130112	3.163429424	2
B7	0.640969728	8.657077719	0.070546609	5.74816113	0.904670438	2.956992205	5
B8	0.377096037	7.347530076	0.058608668	4.000489607	0.751581548	2.164199453	12
B9	0.516059925	9.257689943	0.074155882	5.255840418	0.95095477	2.811966916	7
B10	0.6176142	8.583524786	0.06981134	5.605098886	0.895241553	2.894986818	6
B11	0.216433659	6.14595527	0.048273034	2.825454938	0.619040203	1.602959067	14
B12	0.19007492	3.643603424	0.029087075	2	0.373004707	1.079618416	16
B13	0.437099986	7.454843798	0.059878149	4.345628019	0.767861025	2.309067893	10
B14	0.539948588	7.569410728	0.061527735	4.918166393	0.789014864	2.543276314	9
B15	0.328306718	8.035786128	0.063460462	3.932699273	0.813799626	2.191130031	11
B16	0.321168843	7.20291417	0.057087097	3.666561626	0.732069342	2.020357557	13

, the barriers’ CoCoSo values based on students’ and lecturers’ ratings are presented, respectively. We first focus on the first tier of the barriers. Specifically, from the student’s perspective, the top six barriers are B1, B2, B6, B3, B14, and B7. Meanwhile, from the lecturer’s perspective, the top six barriers are B1, B6, B4, B2, B7, and B10. Such a first glance indicates that the perceptions from the two groups are relatively similar, and barriers, including B1, B2, B6, and B7, received the most attention. However, there are also some diversities between the two groups, such as the students perceiving B3 as a severe barrier while the lecturers perceiving B10 as such.

To fully compare the difference between the perceptions of students and lecturers regarding the severity of barriers, we constructed the rank difference digraph in Figure 2. Specifically, the value of each bar in Figure 2 is equal to the lecturers’ ranks minus the students’ ranks. For example, the rank difference value of B2 is “2” in the digraph, and this value is calculated as 2 = 4 − 2. The “4” represents the lecturer’s perception of B2 as the fourth-ranked barrier, while “2” represents the student’s perception of B2 as the second-ranked barrier. Therefore, if the rank difference value of a certain barrier is positive, it means that such barrier is perceived as having higher severity by students than lecturers, and the negative rank difference value stands for the opposite. The digraph indicates that lecturers and students have achieved an agreement on the perception of the majority of the barriers, as the absolute rank difference values of 12 barriers are only equal to three at most, with most being zero to two. However, there are four barriers whose perceived severity between lecturers and students seems significantly diverse, namely, barriers B3, B4, B10, and B14, as their absolute values of rank difference are all large (i.e., greater than 4). In Figure 2, we use orange colour to highlight these barriers, and use blue colour to represent the barriers whose rank differences are smaller than 4. The next section will discuss their implications.

Figure 2. Barrier rank differences between lectures and students.

Figure 2. Barrier rank differences between lectures and students.

5. Discussion

The above results indicate two main findings. First, students and lecturers both perceive that the barriers B1, B2, B6, and B7 are significant factors negatively influencing the learning outcomes of supply chain management MOOCs. Second, students and lecturers have some diversified thoughts about the severity of the negative impacts of certain barriers, i.e., B3, B4, B10, and B14. In the following paragraphs, we will discuss the implications of the findings and analyse the perception diversifications.

5.1. Strategies of Design and Delivery of Supply Chain Management MOOCs

First, as students and lecturers agreed that barriers B1, B2, B6, and B7 can lead to the most significant negative impacts on MOOC learning outcomes, actions should be taken to mitigate them in the design and delivery of the supply chain management MOOCs. First of all, as B1 represents the lack of connections between MOOC content and supply chain practice, to eliminate this, the design of the MOOCs should add more practical elements. For example, more case studies based on real supply chain operations should be offered, and the introduction of key industrial standards and policies should be added to the delivery of the MOOCs. In the teaching mode, the ways to mitigate the effect of B1 can be the adoption of active learning [52,53], project-based learning [54], and contest-based learning [40].

Also, the B2 indicates that the supply chain knowledge is out of date and cannot give students a good learning outcome. Therefore, the MOOC content should cover the latest knowledge of supply chain management. To achieve this, the MOOCs should iteratively update their curriculums each year by integrating the latest research and industrial progress. For example, MOOCs can invite professional experts to give industrial talks [55]. Also, academicians can be invited to present the new studies. To enable this, the MOOCs can try to create an online discussion forum and invite practitioners and academic staff to it to discuss supply chain management-related questions with the students, allowing the students to benefit from the latest knowledge and practices.

The B6 calls for the attractiveness of the MOOCs. As supply chain management knowledge is partially rooted in operational research, the formulae and mathematical models can be boring for students, which probably reduces the learning motivations and outcomes. Therefore, lecturers should not use the traditional learning-by-listening approach and keep reading slides when delivering the knowledge. Instead, they should attempt to attract and engage students with multiple approaches. For example, lecturers or MOOC designers can provide in-class games to stimulate students’ interests and deliver knowledge [9,56]. Also, multi-media tools, including animations, dynamic graphs, or even virtual reality, can be introduced in the course to make the content more attractive and easier to understand. Moreover, virtual teamwork can be applied [57,58], as students within a team can motivate each other, which can enhance their learning experience when studying boring and difficult staff.

Finally, the B7 indicates the importance of providing necessary and supportive learning resources to enhance the learning outcomes of students. To achieve this, the lecturers can prepare the learning materials from multiple sources. First, the textbooks and references can be provided. Also, other materials such as policy announcements, industrial cases, news reports, or even social media posts related to supply chain management can be offered to make students better understand the domain knowledge and give them chances to apply it to think and solve the practical supply chain issues. Finally, the lecturers can consider providing audio and video materials and integrating them into MOOCs to enhance students’ understanding of the operations of supply chains in reality.

5.2. Comparative Study between Student and Lecturer Perception

One of the novelties of our work is that we compared the different perceptions of barriers between students and lecturers. From our results, we identified that perceptions between the two groups presented diversity in B3, B4, B10 and B14 in Figure 2. Specifically, students think that B3 and B14 are relatively severe but ignore B4 and B10, while lecturers stand on the opposite. According to the student-centred teaching and learning principles [44,45], such diversity can probably pose negative effects on student learning outcomes. This is because the lecturers, not students, are usually responsible for the design and delivery of the MOOCs. Therefore, lecturers can probably ignore the students’ requests on B3 and B14 but put greater effort into B4 and B10. This may fail to solve what really hinders the students’ learning processes.

Therefore, to fix this, lecturers should consider the MOOCs from students’ perspectives and put more effort into the B3 and B14. First, B3 means that if the course content is too easy or too difficult, the MOOCs cannot fit the students’ requirements properly, which can probably demotivate the students and eventually have negative impacts on the learning outcomes. Therefore, from the lecturer’s perspective, to mitigate this effect, they should conduct a thorough course survey about targeted students’ background knowledge about supply chain management before building the curriculum. Based on this, the MOOC content can be tailored to enhance the students’ learning outcomes with proper difficulty [27].

The B14 reveals that fuzzy and ambiguous criteria of learning performance evaluation will harm the learning outcomes of students. Therefore, lecturers should properly design and present evaluation criteria in a clear way, as well as attempt to build personalised evaluation approaches and timely feedback. However, such a key point may be ignored, especially for the lecturer teams with fewer teaching experiences with MOOCs. Therefore, to better solve such a barrier, the lecturers of MOOCs can clearly state the evaluation criteria in each teaching unit of MOOCs instead of only presenting it at the beginning or the end of the entire course. The corresponding feedback should be immediately provided after the completion of each unit so that the students will know their learning outcomes. By doing so, the students can obtain a higher sense of achievement, which stimulates their intention to proceed with learning the subsequent units. To achieve this, the lecturers can send the MOOCs evaluation results to the students’ email (lecturers nowadays can easily automate such a function by cooperating with the MOOCs platform operator) so that the evaluation results can be more effectively delivered, enhancing the students’ learning outcomes. In addition, the evaluation can be conducted with the aid of educational machine learning and statistical analytic techniques [59] so that the evaluation can be timelier and more precise.

The severity of B4 and B10 perceived by lecturers is higher than that of students, possibly meaning that lecturers can adjust their emphasis to the two barriers. On the one hand, B4 states that there is a lack of clear presentation and explanation of the difficult points of MOOCs. It is undoubtful that a clear presentation and explanation of the difficult points can greatly help students, but such a barrier, according to the students’ ratings, may not always have a severe effect on their learning outcomes. Such an interesting finding may reasonably suggest that students have a good ability of self-learning. Therefore, lecturers may not necessarily spend a significantly large amount of time explaining the difficult points in the process of teaching. Instead, they can attach a summative sheet in the learning material files and ask students to read it before taking MOOC lessons. By doing so, they can save time and provide more case studies, quizzes, and group discussions during the sessions to enhance the student learning outcomes from multiple aspects.

On the other hand, compared with the lecturers’ perception, it seems that the students ignore B10 and perceive the discussion forum as not so important. Such a diversified view is interesting and should be interpreted with great caution. In this study, we would argue that it is not because the students do not value the importance of discussion forums. Instead, it may be the consequence that the discussion forum and the relevant activities are not properly designed, leading to the students’ ignorance. In other words, such a finding might occur because of the lecturers’ insufficient experience in organising discussion forums in MOOCs for supply chain management due to their differences from the offline teaching courses. Therefore, the diversified perception of B10 may suggest that the lecturers should enhance their effectiveness in organising discussion forums rather than totally discarding them. For example, the lecturers can ask teaching assistants in discussion forums to guide and support the students’ learning activities, or they can even develop AI teaching assistants based on the ChatGPT or other large language models [60] so that students can benefit more from the discussion forum. Higher education institutions should also provide the necessary financial support for the development of the discussion forums.

6. Conclusions

Supply chain management nowadays is vitally important for enhancing companies’ competitiveness. A good supply chain management needs adequate talent. Due to the development of digital technology and the platform economy, MOOCs have become a popular way of learning supply chain knowledge in China. This paper focused on supply chain management MOOCs and explored the barriers to good MOOC learning outcomes from a Chinese context. The negative online reviews collected from the most widely used Chinese MOOC platforms were analysed, with 16 barriers spotted. After that, the barriers were ranked using a novel TOPSIS-CoCoSo approach, and the solutions to barriers were provided and critically discussed.

The results suggested that the principal barriers were lacking in practical connections, the latest knowledge, attractiveness, and supportive learning materials. Also, we found that the perception of barrier severity between students and lecturers had diversities in the perspectives of inappropriate course difficulty levels, unclear presentation, explanation of difficulty points, insufficient course discussion forum use, and improper design and delivery of course performance evaluation and feedback. We call for a student-centred MOOC design for supply chain management [47].

We believe that this paper has the following academic and practical contributions. On the one hand, this paper summarised barriers to supply chain management MOOCs in a Chinese context. The barriers can reflect the unique properties of Chinese students and lecturers in the digital learning era, which can contribute to the previous educational publication by enriching the understanding of digital learning of supply chain management in China. Also, we provided a comparative study on the barrier perceptions between students and lecturers and located their diversified views. To the best of our knowledge, our work is the first study to explore this direction, offering a new angle for future studies that deeply investigate MOOC teaching and learning in supply chain management or relevant fields. Finally, we proposed a new analytical method by integrating TOPSIS and CoCoSo. Through this approach, the decision makers’ weights can be sufficiently taken into consideration, and the final decision is of great robustness. This newly established approach can provide future educational researchers with new tools for analysing and evaluating course reviews and improving their course quality.

On the other hand, our study has the following practical implications. First, the barriers that we identified and ranked can give the designers of MOOCs in higher education institutions a checklist to update their curriculum, improving course quality on supply chain management and student learning outcomes. Specifically, our results can inform MOOC designers of the necessity of providing the latest supply chain knowledge as well as industrial practices in course content to enhance the students’ learning experiences and outcomes. Also, we confirmed that MOOC content should be attractive, calling for the integration of modern digital tools such as generative AI, virtual reality, and cloud whiteboards into the MOOCs. In addition, learning materials should be developed from multiple sources, making them supportive of students’ learning. Second, our results suggest that there are still cases of MOOC lecturers who lack experience in online teaching and instruction, revealing that higher education institutions should give appropriate training to their lecturers. Also, our study elicits the diversified view between students and lecturers in supply chain MOOCs, indicating the importance of student-centred teaching and learning modes for educational practitioners. Third, the TOPSIS-CoCoSo tool proposed in this paper can support the MOOC platforms’ operations. For example, the tool can be integrated into the MOOC platforms’ recommender systems to better evaluate and recommend courses to learners seeking supply chain management knowledge.

We acknowledge that our study still has the following limitations:

• First, due to our research objective, when we explored the barriers of supply chain management MOOCs, we only considered the Chinese lecturers and learners. However, it should be mentioned that the education system in China is different from that in Western countries [61], meaning that there can be different focuses in MOOCs’ curriculum design, delivery, and performance evaluation;
• In addition, the culture and educational traditions in the Chinese education system have their unique features [62], which can directly or indirectly determine the modes of students’ learning, group-working, and student–lecturer interactions. However, our study has not provided a thorough exploration of such a cultural aspect;
• Also, the MOOC platform we considered in this study is not-for-profit, meaning that the course reviews we collected are all from the MOOCs that are free to the students. However, there are other online learning platforms that charge the learners for subscriptions (such as DataCamp.com, accessed on 5 July 2024). It can probably be that charging fees may lead to different perceptions of learning barriers. Therefore, our results may better fit the scenario for platforms of free MOOCs;
• Finally, we ranked the barriers by their severity of negatively influencing the learning outcomes without considering the interrelationships between the barriers. The reason is twofold. On the one hand, this is because the exploration in evaluating barriers to supply chain management MOOC learning outcomes is in its infancy, and the current research priority of such direction should clarify the most significant barriers to inform the scholars and practitioners. Consistent with the previous literature [63,64], the investigation of the interrelationship between barriers and their compound effects on the learning outcomes of supply chain MOOCs is not covered; otherwise, it can complicate the research design and probably lead to unnecessary interpretation of the results. On the other hand, our method is built on the MCDM framework and rooted in TOPSIS and CoCoSo models; therefore, the interrelationship between factors is not the focus of the mathematical reasoning in our methods and is beyond the scope of the current study;

To address the above limitations and enhance the generalisability of the current study, future research can be conducted in the following perspectives:

• First, to eliminate the limitations posed by the unique features of the Chinese educational system and cultural traditions, the future study can conduct a comparative study. For example, the barriers to supply chain management MOOCs can be investigated in other platforms that are frequently used in Western countries, such as Coursera or MIT open coursework (OCW). By doing so, the potential limitations can be further mitigated, verifying or extending the findings in our study;
• Second, future research can compare the supply chain management MOOCs in the not-for-profit platforms with those commercial learning platforms (e.g., DataCamp.com, accessed on 5 July 2024) and examine if paying tuition fees will lead to different perceptions of barriers to good learning outcomes;
• Finally, future studies can investigate the barrier interrelationships based on our work. For example, they can explore the interrelationship of course-content-related barriers (e.g., interrelationship among B1, B2, B3, B5, and B8), as well as the interrelationship of lecturer-related barriers (e.g., interrelationship among B9, B11, and B12) to investigate how barrier interrelationship can influence MOOC learning outcomes. To conduct such an exploration of barrier interrelationships, methods like structural equation modelling, interpretive structural equation [41], system dynamics [65], and cognitive mapping [66] can be used in future research.