Will Online MOOCs Improve the Efficiency of Chinese Higher Education Institutions? An Empirical Study Based on DEA

Abstract

In recent years, the increasingly fierce competition among higher education institutions (HEIs), the finite resources, and the enormous influence of the COVID-19 epidemic on higher education have made it especially important to evaluate the performance of Chinese higher education institutions. This paper utilizes the DEA-BCC and Malmquist index to analyze the efficiency and productivity of 34 Chinese “985 Project” universities in the period 2017–2021. The indicator system includes three inputs and five outputs, contained in Model 1 and Model 2 for comparative analysis. The results demonstrate that the COVID-19 epidemic has had a considerable negative impact on Chinese higher education, and has induced the reduction of technical efficiency and productivity. Setting up online MOOCs is conducive to enhancing the efficiency and productivity of HEIs; in addition, the efficiency mentioned varies noticeably among different university levels, and there is no significant difference in different university types and geographical locations.

1. Introduction

With a prominent position in education, scientific research, R&D innovation, and social services, HEIs have been highly appreciated by our country and society. In particular, since 2017, the Ministry of Education has promulgated and implemented a number of higher education reform measures, such as the “double first-class” construction, the selection of national first-class courses, the digital transformation of higher education, and the education revitalization in central and western universities. However, HEIs face two external pressures: on the one hand, while demand for higher education is growing, cost reduction remains a priority. These institutions have consumed considerable investment, including sophisticated talents, costly experimental equipment, large numbers of school buildings, and massive annual investment; on the other hand, it is unclear whether the current degree programs in HEIs are meeting the demands of employers. From the perspective of output, the quality of graduates, the level of scientific research achievements, and the contribution to society are also widely of concern in all sectors of society.

At the same time, with the prosperity brought by the internet, various learning platforms and social media platforms have emerged gradually, such as the Chinese University MOOC (https://www.icourse163.org/, accessed on 31 October 2022), Zhihuishu (https://www.zhihuishu.com/, accessed on 31 October 2022), etc. Universities share their courses in the form of videos on these online platforms for students to learn, which not only extends the education scope of HEIs, but also is a kind of social service. Because of the simple and convenient learning style and the feature of enabling repeated learning, this kind of online learning satisfies the self-learning needs of both students and social staff, and has earned considerable popularity. In China, as of November 2022, more than 61,900 MOOCs have been offered, with 402 million registered users and 979 million views, and 352 million undergraduates have earned credits through MOOCs. Meanwhile, MOOCs have also received great attention from the Chinese government, and the Ministry of Education has launched a special task of selecting the National Online Excellent Courses, which is used to assess the quality of MOOCs. Therefore, MOOCs have become significant in Chinese universities and should be taken into account in any study on the current evaluation and ranking of Chinese universities.

In numerous methods of measuring the efficiency of HEIs, data envelopment analysis (DEA) is a suitable research method. Because it is different from the for-profit organization, which aims at maximum profit, the performance of non-profit organizations represented by HEIs needs to be comprehensively calculated from multiple inputs and outputs [1]. DEA is a non-parametric method based on linear programming, which was first established by Charnes and Cooper [2]. It has been widely applied in the performance assessment of universities, hospitals, banks, and comparative analysis between regions.

The previous research on HEIs by DEA mainly concentrated on comprehensive performance [3,4], university ranking [5], teaching performance [6,7], research efficiency [8], and resource allocation [9,10], which evaluated the performance of higher education in many countries and regions from static and dynamic perspectives. For instance, Song [11] researched the productivity of Chinese universities with panel data from 2009 to 2016. Xiong et al. [12] studied resource allocation based on the data from 2009 to 2011. M. Abbott [13] evaluated the teaching and scientific research performance of 36 universities in Australia, and concluded that Australian universities have a high level of efficiency, but the possibility of an overall level of underdevelopment cannot be ruled out. By comparing the performance of Chinese “211 Project” and “non-211 Project” universities, Suthathip Yaisawarng and others [14] explored the impact of China’s higher education reform on the scientific research performance of universities, and concluded that the two types of universities have obvious technical differences. Kortelainen Mika [15] explored the cost structure, efficiency, and productivity performance of higher education institutions in England.

In recent years, studies have explored the performance of higher education in more detail, and the production activities of HEIs have been more clearly described through the multi-stage DEA model [16,17]. For example, Tavares [18] designed a network DEA model and applied it to 45 universities in Brazil, explored the resource allocation and teaching performance of these universities, and also provided reasonable improvement suggestions for inefficient universities. Chonghui et al. [19] used the three-stage DEA method to obtain the true scientific research efficiency of Chinese universities after excluding economic growth, infrastructure, and other environmental factors.

Although past research has reached constructive conclusions, its research theme has certain limitations. Presently, the research on the performance assessment of HEIs in recent years is relatively scarce and does not reflect the education and work characteristics in the context of COVID-19 epidemic. In fact, due to the spread of COVID-19 and the intensification of competition among universities, higher education institutions have had to extend the scale of online education, a feature seldom mentioned in previous studies. Digital transformation is an ongoing process, and the impact of COVID-19 is far-reaching, changing the way people do things, including teaching and learning activities. Therefore, considering the context of COVID-19 epidemic and education’s digital reform, it is worth studying the impact of MOOCs on HEIs’ performance.

This study has three contributions. First, this study enriches the research on the impact of COVID-19 epidemic on higher education. Second, this study discusses the importance of online MOOCs for the efficiency of higher education. Third, this study explored the factors that affect universities’ online MOOC work. Thus, our research seeks to fill the gap in higher education evaluation. Specifically, we took Chinese “Project 985” universities as our research subject. First, we gathered the latest panel data (2017–2021) based on the traditional indicator system to investigate the impact of the COVID-19 pandemic on Chinese higher education; second, we innovatively adjusted the indicator system by introducing factors related to online MOOCs and further compared the results with the former to explore the contribution of the online MOOC to the efficiency and productivity of Chinese higher education. We then analyzed the efficiency differences across different subgroups (university level, university type, and geographical location) to discuss the factors that influence universities’ efforts to conduct MOOCs.

The primary contents of the remaining chapters (as presented in Figure 1) are as follows. The second chapter is a literature review, which sorts out the prior studies and leads to our research motivation. The third chapter introduces the DEA-BCC and Malmquist index, and also explains the indicator system and data used in our study. The fourth chapter is the empirical analysis part: from a dynamic perspective, we analyzed the productivity changes of DMUs over the five years and conducted a comparison between Model 1 and Model 2, and also analyzed the efficiency heterogeneity under three cluster classification rules from a static perspective. The last chapter is a summary and discussion.

2. Literature Review

In this section, we focus on reviewing and summarizing the DEA methods and research themes used in previous studies around the performance assessment by DEA in HEIs, and we categorize them by country.

2.1. Performance Evaluation with DEA in International HEIs

Previous research on the efficiency of foreign HEIs has focused on three main directions: measuring the overall efficiency of higher education, making comparisons of efficiency between subgroups, and exploring the relationship between socially relevant factors and the efficiency of higher education.

In terms of measuring the overall efficiency of higher education, previous studies have generally concluded that the efficiency was relatively high, such as Abbott’s [13] research on Australia. Previous studies have extensively used different models to evaluate the level of higher education: Geraint Johnes [20] used random utility theory and stochastic frontier analysis to compute the cost function of HEIs in UK, while Lee Boon L. [3] developed a network DEA that can capture both quality and quantity to measure the research efficiency of Australian universities. Jill Johnes [21] used the Malmquist productivity index to study the efficiency change of 112 HEIs in UK over the period 1996–2004 and found that the productivity index for each HEI increased at a rate of 1% per annum, largely due to technological progress, while the average technical efficiency over this period was on a downward trend. Similar studies include those of Andersson [22] on the efficiency of HEIs in Sweden and Tavares, and Rafael Santos’s [18] on the evaluation of HEIs in Brazil. In addition, there are studies that focus their perspective on a specific component of higher education, such as earnings efficiency [23], the impact of student mix adjustment on efficiency [15], calculating efficiency from the student’s perspective, and providing improvement options [24].

In terms of efficiency comparisons between subgroups, Johnes [1] examined the efficiency of 109 HEIs in UK in 2000–2001 using an output-oriented DEA-BCC and subdivided these institutions into three categories for efficiency comparisons, concluding that HEIs in the UK generally had a higher input–output conversion ratio over the period and that the differences in efficiency between the three categories were not significant, with significant differences between DMUs with the highest and lowest efficiency scores. Jill Johnes [6] innovatively measured the efficiency of HEIs by considering individual students as DMUs and compared this result with efficiency results derived at a sector-wide level to determine whether inefficiencies in HEIs originated from students or the institution itself.

Other studies have focused on the relationship between societal correlates and higher education efficiency, such as the impact of HEI mergers on efficiency [25,26], concluding that not all HEIs differ significantly in their efficiency levels and that merger activity in HEIs does increase their efficiency levels, although this positive effect is temporary, limited to two or three years after the merger is completed. Other scholars have studied the relationship between regional differences and the efficiency of higher education. Firsova [27] verified the imbalance of higher education quality between different regions in Russia while Agasisti [28] explored the impact of higher education efficiency on the regional economy in Russia and found that high efficiency in higher education has a significant contribution to economic development.

2.2. Performance Evaluation with DEA in Chinese HEIs

According to the dimensions selected for the decision-making unit (DMU), previous studies examining the performance of higher education in China can be divided into two categories: university perspective and regional perspective.

When studies are conducted with regions as DMUs, the expectation is often to obtain efficiency differences between regions (provinces or countries) and furthermore provide policy recommendations for improvement. Relevant studies include those of Wu Jie [29] and Dalai [10] exploring the efficiency of higher education in provinces in China, using methods including a three-stage DEA model and an SBM-DEA model, concluding that higher education levels are higher in the eastern regions and that the differences between DMUs are becoming smaller. Differences in higher education levels between China and other countries have also been studied; for example, Agasisti [30] compared the productivity and technical efficiency differences of ‘elite’ universities in China and Europe.

In a study exploring the efficiency and productivity levels of universities in China, the issue of research efficiency [19] and resource allocation is often the one that has received the most attention. For example, Jill Johnes [31] used DEA-CCR to explore the efficiency of research output of 109 HEIs in China in 2003–2004, and found that universities in coastal areas in China had higher research efficiency, and that comprehensive universities were more efficient than specialized universities. On the basis of Jill Johnes’s work, Jiali Jiang et al. [32] analyzed the current research efficiency of Chinese universities and analyzed the efficiency heterogeneity from three perspectives: university level, geographical location, and university type. Guo-liang Yang [33] used a two-stage network DEA model to measure the efficiency of research institutions in China. An Qingxian [34] constructed an additive DEA model to measure the research and service efficiency of 38 “Project 985” universities in China. Song Yaoyao [11] studied the productivity levels of 58 universities in China during the period 2009–2016. In particular, Xi Xiong [12] used a parallel-DEA approach to provide a theoretical basis for the allocation of educational resources to universities.

2.3. Research Gap

To the best of our knowledge, few studies have included MOOCs in the context of performance evaluation, and there is a lack of research on the impact of MOOCs on university performance. Therefore, our study attempts to fill this gap.

3. Methodology and Data

3.1. Methodology

Tommaso Agasisti [30] has verified the rationality of the variable return-to-scale hypothesis with statistical test methods, so our research also takes a variable return-to-scale as the premise. At the same time, because the input of higher education is not entirely controlled by the universities themselves, the evaluation of the efficiency of universities should be based on the output-oriented perspective and attach importance to the maximization of output. In the following research, we primarily used the output-oriented DEA-BCC and Malmquist index. Their theoretical computation methods are as follows.

3.1.1. DEA-BCC

Technical efficiency is used to evaluate the degree to which the production process of a DMU reaches the technical level of its industry, reflecting its ability to transform input into output. That is, in the context of output orientation, given the input level, the higher the output level, the higher the technical efficiency.

Suppose we want to measure the technical efficiency of n DMUs, recorded as DMU_j (j = 1, 2, …, n): each DMU has m inputs, which are recorded as x_i (i = 1, 2, …, m), and the weight of inputs is v_i (i = 1, 2, …, m). There are q outputs, which are recorded as y_r (r = 1, 2, …, q), and the output weight is u_r (r = 1, 2, …, q). Record the DMU to be considered as DMU_k, and its output-oriented efficiency (Eff) under CRS hypothesis can be calculated as:

$$\begin{array}{cc}{Eff}_k=\hfill & \underset{v,u}{min}\frac{{\sum }_{i=1}^m{v}_i{x}_{ik}}{{\sum }_{r=1}^q{u}_r{y}_{rk}}\hfill \\ s.t.\hfill & \frac{\sum _{r=1}^q{u}_r{y}_{rj}}{\sum _{i=1}^m{v}_i{x}_{ij}}\le 1\hfill \\ & v,u\ge 0.\hfill \end{array}$$

(1)

Of course, the current programming model is not a linear programming model, so we need to convert it into a linear programming model through the Charnes-Cooper transition and then we have the following model:

$$\begin{array}{c}\underset{x}{min}{\displaystyle \sum _{i=1}^m}{\nu }_i{x}_{ik}\\ s.t.{\displaystyle \sum _{r=1}^s}{\mu }_r{y}_{rj}-{\displaystyle \sum _{i=1}^m}{\nu }_i{x}_{ij}\le 0\\ {\displaystyle \sum _{r=1}^q}{\mu }_r{y}_{rk}=1\\ \nu \ge 0;\mu \ge 0\\ i=\mathrm{1,2},\dots ,m;r=\mathrm{1,2},\dots ,q;j=\mathrm{1,2},\dots ,n\end{array}$$

(2)

where $t=\frac{1}{{\sum }_{i=1}^m{v}_i{x}_{ij}}$,$\mu =tu$, $\nu =tv$.

Then, the dual of model (2) can be written as follows:

$$\begin{array}{c}\mathit{max}\phi \\ s.t.{\displaystyle \sum _{j=1}^n}{\lambda }_j{x}_{ij}\le x_{ik}\\ {\displaystyle \sum _{j=1}^n}{\lambda }_j{y}_{rj}\ge \phi y_{rk}\\ \lambda \ge 0\\ i=\mathrm{1,2},\dots ,m;r=\mathrm{1,2},\dots ,q;j=\mathrm{1,2},\dots ,n.\end{array}$$

(3)

Add the constraint of ${\sum }_{j=1}^n{\lambda }_j=1$, and we can obtain the output-oriented BCC model [35], which can be expressed as:

$$\begin{array}{c}\mathit{max}\phi \\ s.t.{\displaystyle \sum _{j=1}^n}{\lambda }_j{x}_{ij}\le x_{ik}\\ {\displaystyle \sum _{j=1}^n}{\lambda }_j{y}_{rj}\ge \phi y_{rk}\\ {\displaystyle \sum _{j=1}^n}{\lambda }_j=1\\ \lambda \ge 0\\ i=\mathrm{1,2},\dots ,m;r=\mathrm{1,2},\dots ,q;j=\mathrm{1,2},\dots ,n.\end{array}$$

(4)

3.1.2. Malmquist Index

In order to evaluate the productivity change of DMU in a continuous period and the impact of technical efficiency and technological progress on productivity change, we normally use the Malmquist index for analysis. This concept originated from the research of Malmquist (1953) [36]. Färe R et al. (1992) [37] first incorporated the Malmquist index into the calculation of DEA and decomposed the Malmquist index into two aspects: the technical efficiency change of DMU in two periods, and the technological progress that portrays the change of technology catch-up.

Figure 2 displays the performance of a DMU in two periods in the input-oriented CRS labeled K1 and K2, respectively. A1B1C1 represents the production frontier of period 1, and A2B2C2 represents the production frontier of period 2.

When calculating the efficiency with the frontier of A1B1C1 (period 1) as the production frontier, K’s Malmquist productivity index can be expressed as:

$$E_1\left(K_1\right)=\frac{O{K}_1^{\prime }}{O{K}_1},E_1\left(K_2\right)=\frac{O{K}_2^{\prime }}{O{K}_2},$$

(5)

where $E_1\left(K_1\right)$ denotes the true efficiency of K in period 1 and $E_1\left(K_2\right)$ denotes the efficiency of K in period 2 relative to period 1’s frontier.

When calculating the efficiency with the frontier of A2B2C2 (period 2) as the production frontier, K’s Malmquist productivity index can be expressed as:

$$E_2\left(K_1\right)=\frac{O{K}_1^{″}}{O{K}_1},E_2\left(K_2\right)=\frac{O{K}_2^{″}}{O{K}_2},$$

(6)

Similarly, $E_2\left(K_2\right)$ denotes the true efficiency of K in period 2 and $E_2\left(K_1\right)$ denotes the efficiency of K in period 1 relative to period 2’s frontier.

According to the method of calculating the Malmquist index by Caves et al. (1982) [38], the geometric mean of the two Malmquist indexes is used as the Malmquist index of the evaluated DMU; that is:

$$M\left(K_2,K_1\right)=\sqrt{\frac{E_1\left(K_2\right)\ast E_2\left(K_2\right)}{E_1\left(K_1\right)\ast E_2\left(K_1\right)}}.$$

(7)

Thus, the Malmquist index from period t to period t + 1 can be expressed as:

$$M\left(x_{t+1},y_{t+1},x_t,y_t\right)=\sqrt{\frac{E_t\left(x_{t+1},y_{t+1}\right)\ast E_{t+1}\left(x_{t+1},y_{t+1}\right)}{E_t\left(x_t,y_t\right)\ast E_{t+1}\left(x_t,y_t\right)}}.$$

(8)

Assume that the technical efficiency of DMU in two periods is $E_t\left(x_t,y_t\right)$ and $E_{t+1}(x_{t+1},y_{t+1})$, then the change of technical efficiency can be calculated by the ratio of them:

$$EC=\frac{E_{t+1}\left(x_{t+1},y_{t+1}\right)}{E_t\left(x_t,y_t\right)}.$$

(9)

After the change of technical efficiency is separated from the Malmquist index, the remaining part is the catch-up of the technical frontier. When TC is less than 1, the frontier moves backward, while when TC is greater than 1, the frontier moves forward, which also means technical progress. The above can be stated as follows:

$$TC=\sqrt{\frac{E_t\left(x_t,y_t\right)\ast E_t\left(x_{t+1},y_{t+1}\right)}{E_{t+1}\left(x_t,y_t\right)\ast E_{t+1}\left(x_{t+1},y_{t+1}\right)}}.$$

(10)

3.2. Data

We take Chinese “985 Project” universities as a case study. These universities are under the management of the central government directly, and have high attainments in education and academic level. At the same time, they also participate in social service and transfer large amounts of technology every year. All of the above have reflected their leadership in Chinese higher education. Therefore, the performance evaluation of such universities can reflect the latest development level of Chinese higher education to a great extent.

In terms of sample size, there are 39 “985 Project” universities in China. However, five of them are directly affiliated with the Ministry of Industry and Information Technology or the Central Military Commission, so the related information is not wholly issued for confidentiality reasons, which means our number of DMUs has to be decreased to 34. From the perspective of the time dimension, we selected 2017–2021 as our research period, which not only includes the outbreak of COVID-19 in 2020, but also is in the rapid development period of education digital reform, so it can support us in exploring the influence of COVID-19 on Chinese HEIs and the impact of online MOOCs on university efficiency.

In summary, we collected data from 34 universities in China for the period of 2017–2021, which involved eight indicators, from each university’s information disclosure website as well as from the National Smart Education Platform (https://www.smartedu.cn/home/province?name=%E9%AB%98%E6%95%99, accessed on 31 October 2022). The detailed indicators and their sources are presented in the next subsection.

3.2.1. Indicators

In order to assess efficiency as objectively as possible, the indicator system used in this study covers as comprehensively as possible the various production characteristics of a university. We drew on indicators that have been shown to be adequate in previous studies, while also incorporating relevant features of MOOCs in order to reflect the innovation of this study. We eventually selected three inputs and five outputs, and divided them into Model 1 and Model 2 for comparative analysis.

Table 1. The indicators used in this paper.

Model	Input	Output
Model 1	Number of full-time teachers (NOFTT) Total number of students (TNOS) Financial budget (FB)	Number of patent applications (NOPA) Employment rate of undergraduates (EROU) Number of core papers published (NOCPP)
Model 2	Number of full-time teachers (NOFTT) Total number of students (TNOS) Financial budget (FB)	Number of patent applications (NOPA) Employment rate of undergraduates (EROU) Number of core papers published (NOCPP) Number of online MOOCs run (NOMO) Playback times of online MOOCs (PTOOM)

presents our models and indicator system. In the input factors, we have comprehensively considered the human and financial costs; in the output factors, we included online MOOCs and the three functions of universities. Therefore, our indicator system possesses a certain integrity.

The number of full-time teachers (NOFTT) is compiled in the Annual Undergraduate Teaching Quality Report of each university, including the total number of personnel engaged in teaching and scientific research. Among them, teachers involved in teaching can reflect a university’s human investment in education, which can conform to the employment rate of undergraduates and the number of online MOOCs in the output factors. Teachers engaged in scientific research can reflect the human input of a university in scientific research and social service, which echoes the number of papers and patents among the output factors. Ideally, we should use the two as separate input indicators to measure the efficiency of each DMU. However, due to the incomplete disclosure of relevant data, our research is hindered from further improvement.

The total number of students (TNOS) is also collected in the Annual Undergraduate Teaching Quality Report. In some previous studies, this indicator was regarded as an output variable to measure the ability of a university to cultivate students. However, in our research, we regard it as an input variable, chiefly because the students on campus include undergraduates, and master’s and doctoral students. As more and more undergraduates directly participate in scientific research projects, our total number of students on campus can also be regarded as the human cost of scientific research in universities.

The financial budget (FB) is compiled from the Annual Department Budget, which is composed of three parts: the first part is the government’s scientific education appropriation, which is used to maintain the daily education and operation of colleges and universities. The second part is the fund subsidies of government and committees to support the scientific research work of universities. The third part comes from the revenue of enterprises and society, such as the horizontal projects of scientific research. The financial budget reflects the financial input of HEIs, a common input indicator in previous studies.

The number of patent applications (NOPA) is gathered in a specialized patent database in China (https://patents.qizhidao.com, accessed on 31 October 2022). We searched and collected the panel data of this indicator according to the university name and time. The number of patent applications is a significant aspect of measuring a university’s ability to serve society, marking the conversion of scientific research from theory to practice. With the purpose of “more is better”, we regard it as an output variable.

The employment rate of undergraduates (EROU) is collected from the Annual Employment Quality Report of Graduates. In the existing research, the number and graduation rate of graduates are usually used to measure educational achievements, but such indicators cannot reflect the quality level of graduates. The employment rate itself contains a degree of recognition from society and enterprises on the ability of graduates. The higher the employment rate, the higher the comprehensive ability of the students we will think. Especially in the current epidemic situation, the employment rate of college graduates is particularly valued. Therefore, we regard it as an output indicator, which also conforms to the current social background.

The number of core papers published (NOCPP) is gathered from the core collection database, Web of Science, filtering by university name and time. Since the data are all from the identical platform, we are assured of the data comparability of each DMU. As we all know, the paper is a summary and presentation of research work, and the quantity and quality are equally important. In this research, we filtered out some papers in ordinary journals and only focused on the number of high-quality papers published, which also represents the substantive research level of universities.

The number of online MOOCs offered (NOMO) and the playback times of online MOOCs (PTOOM) are collected from an official online comprehensive MOOC platform, the National Smart Education Platform, which includes several subplatform courses, such as the Chinese University MOOC (https://www.icourse163.org/, accessed on 31 October 2022), Xuetangzaixian (https://www.xuetangx.com/, accessed on 31 October 2022) and Zhihuishu (https://www.zhihuishu.com/, accessed on 31 October 2022). We still searched by time and university name, and summarized the panel data of online MOOCs of universities through the data crawler by Python. Because several subplatforms did not match our query conditions and the number of courses was small, we did not include them in our statistical scope. The number of online MOOCs well reflects the level of digital education in HEIs, and also reflects the sensitivity of HEIs to the epidemic. Therefore, the inclusion of the above two indicators into the indicator system also reflects the timeliness of current higher education. It is worth noting that the number of online MOOCs run and the playback times of online MOOCs in some universities is 0. In order to facilitate the subsequent DEA analysis, we replaced these 0 values with 0.1, but this does not affect the ultimate efficiency results.

3.2.2. Descriptive Statistics

We have conducted a summary collation of the data for macro analysis. Descriptive statistics of each indicator are described in Appendix A. In the number of full-time teachers, the total number of students, and other indicators, the maximum value of DMU is more than three times the minimum value. Further, among the indicators such as the number of patent applications, the number of core papers published, and the online courses, the gap is much greater, even a tenfold gap. These data show that although both of them belong to the “985 project” universities, the difference between them is still huge, and the hidden reason behind this huge difference is most probably the inefficiency of DMU.

In addition, Figure 3 displays the growth trend of indicators. On the whole, the data demonstrated an increasing tendency over five years, with a significant growth rate, but some indicators showed a downward trend. For example, the number of full-time teachers diminished in 2018 and then rose steadily. The total number of students in school maintained an average annual growth rate of 2.9%, which was accompanied by the growth of financial pressure, teaching pressure, and employment pressure. The financial budget achieved a breakthrough increase, from 773,615.092 yuan to 1,016,061.458 yuan, with a growth rate of more than 30%. The number of patent applications shows a different trend from the above three: since 2020, the number of patent applications has shown a downward trend, which may be related to the outbreak of the COVID-19 epidemic. Also affected by the epidemic was the employment rate of undergraduates, which performed the worst in five years in 2020. The number of core papers published increased from 5363 to 9330, with an average annual growth rate of 14.8%. The number of online courses increased year by year from 2017 to 2020, but decreased significantly in 2021, which may be associated with the slow recovery of face-to-face teaching. Finally, the number of online courses offered increased rapidly from 324,234 to 857,610, realizing growth of nearly three times, especially in 2020 and 2021.

Through the descriptive statistics of the data, we can draw the following conclusions. (1) In the past five years, most of the input and output factors have achieved substantial growth, so China’s higher education is progressing in a good direction as a whole. (2) The trend of indicators related to online MOOCs reflects that HEIs are paying more and more attention to online education. (3) The COVID-19 epidemic has considerably influenced scientific research and higher education.

4. Empirical Results

In this chapter, we take 34 “985 Project” universities as DMUs, such as Peking University, Tsinghua University, Zhejiang University, etc. Based on the indicator system and data presented in the previous chapter, we use the Malmquist index to evaluate the productivity changes of Model 1 and Model 2 from 2017 to 2021, and use output-oriented BCC to investigate the efficiency differences of different groups. As efficiency comparison is involved, each section utilizes the Kruskal–Wallis test [39] to determine whether the difference is significant. Based on the empirical results, we provide feasible explanations.

4.1. Dynamic Analysis

Based on the collected data, we used DEAP2.1 to measure the productivity of Model 1. Since there are no relevant indicators of online MOOCs in Model 1, we obtained the results of a traditional university performance evaluation, which also extends the prior research on the efficiency of Chinese HEIs at the time level.

Table 2. The Effch, Tech, and Tfpch’s values of Model 1.

Period	Effch	Tech	Pech	Sech	Tfpch
2017–2018	0.976	1.078	0.996	0.979	1.052
2018–2019	1.008	1.048	0.994	1.013	1.056
2019–2020	1.030	0.928	0.989	1.042	0.956
2020–2021	0.975	1.075	0.995	0.981	1.049
Mean	0.997	1.030	0.993	1.003	1.027

displays the comprehensive performance of each DMU over five years. The development trend of technical efficiency, technical progress, and productivity are shown in Figure 4. First of all, from the perspective of productivity, the average productivity over five years was 1.027, which indicates that the performance of Chinese HEIs has shown an overall upward trend in five years, with an average annual growth rate of 2.7%. However, there is also a caveat to the high productivity; that is, productivity was relatively volatile, exhibiting an “N” shaped trend of change, showing a decline and growth in 2020 and 2021, respectively. Due to the impact of COVID-19 in 2020, the work of HEIs was blocked, and their efficiency was seriously reduced. After a year’s adaptation, universities gradually recovered their former efficiency level, and this “adaptation” is what we are fascinated by most. In addition, since productivity is derived from comprehensive technical efficiency and technological progress, we need to analyze the various components of productivity deeply. The five-year average performance of comprehensive technical efficiency was 0.997, which indicates that the efficiency of DMUs in transforming input into output declined, which can also be inferred from the low pure technical efficiency. The technical progress demonstrated strong vitality over five years, providing a 3% growth in productivity, which shows that universities in China paid attention to the application of new technology in education and operation work in the five years.

The efficiency and productivity results of each DMU are shown in Appendix A. The productivity of 17 DMUs exceeded the average value, accounting for 50%. Shanghai Jiao Tong University ranked first with productivity of 1.125, while the Minzu University of China and Ocean University of China had the lowest productivity score of only 0.94. The difference between the highest score and the lowest score was 0.185, which shows that the development of China’s educational resources among universities is not balanced. Taking the Minzu University of China as an instance, its low productivity was entirely due to lack of technological progress. Therefore, it should pay more attention to the application of advanced technologies to teaching and scientific research to improve productivity.

At the same time, among 34 DMUs, 18 had a comprehensive technical efficiency score equal to or greater than 1, and the remaining DMUs were in the invalid rate state. In order to realize the conversion of a DMU from low efficiency to high efficiency, the improvement direction of each DMU is different. For example, the comprehensive technical efficiency, pure technical efficiency, and scale efficiency of Tsinghua University were 0.95, 1, and 0.95, respectively, which reveals that the current input–output transformation of Tsinghua University was efficient, and the reason for the low comprehensive technical efficiency was the low scale efficiency. Therefore, Tsinghua University should appropriately expand its school scales, taking steps such as introducing more students, teachers, and researchers. The corresponding indicators of Wuhan University were 0.989, 0.976, and 1.014, respectively, which shows that the current low pure technical efficiency of Wuhan University caused its low comprehensive technical efficiency. Consequently, its existing input–output structure should be adjusted to achieve a reasonable allocation of various inputs.

On the basis of Model 1, “the number of online courses” and “the number of online courses played” were added as input elements to re-measure the performance of each DMU in Model 2. We nevertheless present the performance of the time dimension and DMU dimension, respectively. The results are as follows.

Table 3. The Eff, Tech, and Tfpch’s values of Model 2.

Period	Effch	Tech	Pech	Sech	Tfpch
2017–2018	0.996	1.269	0.996	1	1.263
2018–2019	1.014	1.154	0.996	1.018	1.169
2019–2020	1	1.033	0.99	1.011	1.033
2020–2021	0.987	1.029	0.997	0.99	1.015
Mean	0.999	1.117	0.994	1.005	1.116

and Figure 5 manifest the productivity and technical efficiency changes of Model 2 over five years. It can be found that after adding online MOOC indicators, all performances improved: productivity increased from 1.027 to 1.116, comprehensive technical efficiency rose from 0.997 to 0.999, and technical progress rose from 1.030 to 1.117. These results show that against the complex social background of the current epidemic situation, the online higher education is an effective measure to improve technical efficiency and productivity. At the same time, the productivity was above 1, which means HEIs have paid more attention to online MOOCs, and the decrease year by year may be due to the mature nature of the work of universities, which is also consistent with the phenomenon of online teaching in China during COVID-19.

From a micro perspective, the performance of DMUs in Model 2 is shown in Appendix B. In Model 2, the productivity of 17 DMUs exceeded the average value. Beijing Normal University ranked first with an efficiency value of 1.309, and Lanzhou University ranked last with an efficiency value of 0.975. Although the gap between the maximum and minimum was still large, the minimum of Model 2 was closer to 1 than that of Model 1, indicating that universities have made efforts in online MOOCs. Specifically, Lanzhou University should pay more attention to the rational allocation of existing input resources to achieve the improvement of productivity, because its low comprehensive technical efficiency is the fundamental reason for low productivity.

In terms of comprehensive technical efficiency, there were 25 DMUs with efficiency greater than or equal to 1, 7 more than identified by Model 1. It is worth noting that Tsinghua University, Peking University, and others switched from the ineffective rate in Model 1 to the efficient rate, which shows that these universities indeed put more resources and energy into the output of online MOOCs, which is unobservable in Model 1.

In order to further enhance the effectiveness of data comparison, we conducted analysis to determine whether there was a statistically significant difference between the results of the two models. Since the data themselves did not conform to the normal distribution, we utilized the Kruskal–Wallis test for analysis. The inspection results are shown in

Table 4. Kruskal–Wallis test results.

Indicators	Kruskal-Wallis H (K)	df	p-Value
Tfpch	21.273	1	0.000004
Eff	0.432	1	0.511
Tech	21.956	1	0.000003

.

The test results are consistent with what we anticipated: the productivity and technological progress of the two models are significantly different, while the comprehensive technical efficiency is not significantly different. Therefore, we are more confident that in the current social context, the development of online education is indeed a viable solution to enhance efficiency, and this process is chiefly achieved by technological progress; that is, the adequate introduction of new technologies, including online MOOC platforms.

4.2. Static Analysis

In order to further explore the factors that affect the sensitivity of HEIs to carry out online MOOC work, we considered taking the sectional data of Model 2 in 2021 as the dataset, analyzing the three dimensions of school level, school type, and geographical location, and using the output-oriented BCC model for data analysis.

According to the classification of universities by the Ministry of Education, 34 DMUs were divided into three groups: A, B, and C. Group A represented C9 universities, Group B included other “double first-class A” universities, and Group C included “double first-class B” universities. The experimental results are shown in

Table 5. The Eff, Tech, and Tfpch’s values of DMUs at different university levels.

Group	Crste	Vrste	Scale
A	0.9735	0.988875	0.983125
B	0.865435	0.966913	0.89213
C	0.914667	0.941667	0.97

. According to the experimental results, the comprehensive technical efficiency of the three groups was ranked as A > C > B. Only one of the eight samples in Group A was in the invalid rate state, which indicates that the conversion efficiency from input to the output of C9 universities is in the dominant position in China, and its high pure technical efficiency also proves this opinion. Although the universities in Group C are “double first-class B” universities, it seems that it is precisely for this reason that their technical efficiency exceeds that of the universities in Group B. As the main body of “985 project” universities in China, the low efficiency of Group B universities can be explained as follows: on the one hand, there are many universities with various types, which inevitably lead to uneven quality. On the other hand, according to the data itself, the low technical efficiency was principally due to the absence of scale efficiency, which reminds Group B universities to pay attention to expanding their scale. They can expand their output by increasing the number of online MOOCs. After all, their pure technical efficiency is considerable.

Furthermore, the result of the Kruskal–Wallis test is shown in

Table 6. The Kruskal–Wallis test results from different university levels.

Kruskal–Wallis H (K)	df	p-Value
5.427	2	0.066

, indicating a significance at the 10% level. Therefore, we believe that there are significant differences in technical efficiency among universities at different levels. One of the possible reasons for this result is that the higher-level nature attracted premium students and research strength, which is more conducive to obtaining more and better resources and producing more results. For the purpose of sprinting towards the “double first-class A” goal, the universities in Group C have shown extraordinary efficiency, as well.

Jill Johnes’s (2008) research [8] on HEIs in China shows that comprehensive universities have higher efficiency than professional universities. In order to investigate whether this conclusion is also applicable to the consideration of the comprehensive performance of higher education institutions and the current era background, we divided the 34 DMUs into comprehensive universities and professional universities.

Table 7. The Eff, Tech, and Tfpch’s values of DMUs in different university types.

University Type	Crste	Vrste	Scale
Comprehensive university	0.8865	0.965346	0.915
Professional university	0.9235	0.9845	0.938

displays experimental results. As the result indicates, the technical efficiency of professional universities seems to be higher, but the Kruskal–Wallis test (as seen in

Table 8. The Kruskal–Wallis test result from different university types.

Kruskal–Wallis H (K)	df	p-Value
0.630	1	0.428

) tells us that there is no significant difference between the two groups of data. Therefore, China’s comprehensive “985 Project” universities and professional “985 Project” universities are at the equivalent level in terms of input–output conversion efficiency.

In addition, in order to examine the efficiency differences of universities in different regions, we divided 34 DMUs into four parts: east, middle, west, and northeast. The results are shown in

Table 9. The Eff, Tech, and Tfpch’s values of DMUs in different areas.

Region	Crste	Vrste	Scale
East	0.9022	0.97885	0.91975
Middle	0.892	0.9554	0.9298
West	0.908667	0.9635	0.942
Northeast	0.827	0.946667	0.866

. The average technical efficiency of the east, middle, and west was roughly the same, but the technical efficiency of the northeast was significantly lower than that of the other three areas. The Kruskal–Wallis test (as can be seen from

Table 10. The Kruskal–Wallis test result from different university locations.

Kruskal–Wallis H (K)	df	p-Value
0.548	3	0.908

) tells us that there was no significant difference in technical efficiency between the four regions.

This section compares the efficiency differences between subgroups from three perspectives, and finds that the technical efficiency differences caused by different school levels were the most obvious, while the technical efficiency differences caused by school types and geographical locations were not significant. Since our indicator system includes relevant indicators of online MOOCs, we believe that external factors such as school type and geographical location will not restrict the positive performance of DMUs in online MOOCs. In contrast, internal driving factors such as the strength of teachers and students of the school itself are crucial factors affecting technical efficiency.

5. Conclusions, Implication, and Limitations

5.1. Conclusion and Discussion

The importance of HEIs’ performance assessment stems from its public capital investment and powerful social influence. For China, a country with a large population that is relatively short of educational resources, the importance of this performance evaluation is particularly prominent. Coupled with the impact of COVID-19 since 2020, the increase in online higher education has become an inevitable trend. On the basis of previous studies, we have made innovative adjustments to the input and output indicator system, incorporating online MOOC-related factors into the indicator system, and taking into account the performance of universities in the latest period. In addition, we also ran two models to research the sensitivity of Chinese universities to the epidemic and online education, and observed the efficiency differences in subgroups from the perspective of university level, university type, and geographical location. Our research is based on the performance of 34 “985 Project” universities in 2017–2021, and draws the following conclusions.

(1) From the results of the dynamic analysis, China’s “985 Project” universities have shown excellent performance from a conventional perspective (Model 1) during the period 2017–2021, with an average annual increase rate of 2.7% in productivity, which manifests that the Chinese higher education level has been improved during this period. However, from the perspective of development trends, productivity and technological progress have shown some volatility. The volatility of comprehensive technical efficiency was more obvious, and it also appears to have declined in 2020, with an average efficiency of 0.997, declining at a rate of 0.3%. After adding two indicators of online MOOC into our research (Model 2), we find that both productivity and technical efficiency achieved a certain growth, especially the productivity growth of 8.9%. Furthermore, the result of the Kruskal–Wallis test verifies the significant difference between the two. (2) From the results of static analysis, the efficiency difference caused by the university level was significant: C9 universities had the highest comprehensive technical efficiency, followed by the “double first-class B” universities, and finally the other “double first-class A” universities. However, the subgroup efficiency of university type and geographical location did not show a significant difference.

In conclusion, our research confirms that the impact of the COVID-19 epidemic on Chinese higher education has been huge, downgrading the productivity and efficiency of HEIs, and affecting the normal input–output transformation. However, Chinese universities have made full use of the advantages of advanced technology and have great sensitivity to cope with unpredictable challenge. At the same time, online education, such as online MOOCs, is a desirable way to improve efficiency, and this digital transformation is determined by the internal drive of HEIs themselves, regardless of university type and geographical location. Consistent with Kai Wang’s research findings [40], our research also recognizes the important role of MOOCs, and considers that expanding the number and quality of MOOCs is an effective way for universities to improve efficiency.

5.2. Implications

The digital development of education due to the COVID-19 pandemic represents a new beginning, and our study provides interesting hints about the importance of online MOOCs for HEIs’ efficiency. Both policymakers and higher education institutions could benefit from the results. Based on the research conclusions, we put forward several policy implications.

This study found that the COVID-19 epidemic has had a relatively large negative impact on Chinese HEIs. HEIs are highly sensitive to the COVID-19 epidemic, and promoting and expanding online MOOC courses will help improve efficiency. Hence, after experiencing the negative impact of COVID-19, referring to the results of Appendix C, inefficient universities should follow the corresponding benchmark ones and draw lessons from their resource allocation model to improve their own efficiency. At the same time, the department of higher education should vigorously support universities to carry out online MOOC work, launch more award and evaluation policies based on the existing “national online high-quality course” evaluation, properly increase the proportion of MOOCs in the “double first-class” evaluation, and actively publicize high-quality online MOOCs to society to improve the overall level of education in China.

This study points out external factors such as school type and geographical location will not restrict the positive performance of each DMU in online MOOCs, but the school’s own teacher–student strength and internal drive are the key to affecting technical efficiency. Therefore, suggestions can be given regarding the internal driving forces of colleges, teachers, and students. From the perspective of colleges and universities, they should improve the construction mechanism of MOOC courses and set up a professional management team for management. Colleges and universities should also give full play to their own subject characteristics, create a group of online MOOC courses with their own professional advantages and characteristics, improve the quality of online MOOC courses, and improve college teaching and social service capabilities. From the perspective of teachers, they should change the concept of teachers, build a high-quality teaching team, adapt to emerging teaching concepts and technologies, and innovate in combination with their own teaching, let emerging technologies serve teaching practice, and improve their own teaching quality in practice. From the perspective of students’ initiative, they should improve students’ awareness of MOOCs, make full use of student groups, and publicize and promote MOOCs.

5.3. Limitations

This paper aimed to evaluate the performance of Chinese universities in recent years through a complete system covering education, scientific research, and social services. However, due to the restriction of data collectability, some potential unstructured data that is not easy to collect (for example, text-based data) may have been overlooked, thus the paper may lack a more complete evaluation of the overall efficiency. Therefore, some non-structural data that can reflect university performance can be included in the indicator system in future research. Another future research direction to be considered is expanding the size of research samples, including Chinese “211 Project” universities and additional excellent universities in the research system so as to more representatively reflect the level of Chinese higher education. In addition, future research can also consider the application of novel DEA methods to higher education performance assessment, such as the combination of DEA with artificial intelligence methods [24,41,42] and DEA with game theory [43] and PCA [44].