Preventive Risk Management of Resource Allocation in Romanian Higher Education by Assessing Relative Performance of Study Programs with DEA Method
Round 1
Reviewer 1 Report
1. The introduction does not show a straightforward picture where the author focuses too much on the education sector's importance for a country's progress, while it is a commonsense thing. I suggest that from the first paragraph the author focuses directly on the dilemma of pursuing performance and the issues of efficiency and risk. This does not mean that the importance of the education aspect is ignored, but it is sufficient to mention it briefly.
2. I find some quotes are too old. To show the quality of the paper, the author should refer to quotes from the last 10 or even 5 years. Moreover, the issue of risk management in the HEI sector is indeed a new focus in the literature
3. The highlighting of the research gap is not so obvious, it needs to be raised explicitly so that readers can quickly grasp the novelty of this research.
4. The rationale for using the DEA approach needs to be strengthened.
5. “To avoid distortions in the 73 efficiency scores, we decided to implement the bootstrap method to correct DEA 74 efficiencies.” This claim needs a citation
6. In the introduction, it is necessary to add practical, theoretical, and literary contributions from the results of the study
7. Many statements that I think need citation support. Better to do
8. In the conclusion section, it is best to remind yourself of the purpose of this research, before the author submits a summary of the findings.
9. It would also be better at the end of the paper to highlight the limitations of the research
Author Response
Thank you for the useful remarks. We have included modifications to your requests in the revised paper. Below there are punctual answers to your observations.
- The introduction does not show a straightforward picture where the author focuses too much on the education sector's importance for a country's progress, while it is a commonsense thing. I suggest that from the first paragraph the author focuses directly on the dilemma of pursuing performance and the issues of efficiency and risk. This does not mean that the importance of the education aspect is ignored, but it is sufficient to mention it briefly.
Answer: This issue is treated now in the first paragraph of the paper.
- I find some quotes are too old. To show the quality of the paper, the author should refer to quotes from the last 10 or even 5 years. Moreover, the issue of risk management in the HEI sector is indeed a new focus in the literature
Answer: We have searched in ISI Web of Science on the use of DEA in higher education. There is a lack of recent publications in this area, and no related publication about the Romanian case. The references that look to old are mostly related to the theory of DEA, which is quite a mature method, developed some tens of years ago. Therefore, the list of references includes quotes older than 10 years. We have added in the list of references a new paper, dated 2015.
- The highlighting of the research gap is not so obvious, it needs to be raised explicitly so that readers can quickly grasp the novelty of this research.
Answer: This issue was tackled in the revised version – lines 80 ….
- The rationale for using the DEA approach needs to be strengthened.
Answer: This issue was tackled in the revised version – lines 87 …
- “To avoid distortions in the 73 efficiency scores, we decided to implement the bootstrap method to correct DEA 74 efficiencies.” This claim needs a citation
Answer: We have cited this issue.
- In the introduction, it is necessary to add practical, theoretical, and literary contributions from the results of the study
Answer: At the end of the introduction we highlight some core results of the study in the revised version.
- Many statements that I think need citation support. Better to do
Answer: We have revised the text and added citations in several places.
- In the conclusion section, it is best to remind yourself of the purpose of this research, before the author submits a summary of the findings.
Answer: This request was considered.
- It would also be better at the end of the paper to highlight the limitations of the research
Answer: This request was considered.
Reviewer 2 Report
The article propose a helpful approach for the university management, interested in reducing risks. It is used a reliable model of evaluating how efficient study programs can be while using various resources, so university management can compare their different study programs, within the same university or nation-wide.
However, there are some points of inquiry about the research design used, and aspects considered as inputs and outputs, as it was not explained why they are the most relevant ones, and there were used only them. Why, for instance, square meters are so relevant, mainly the pandemic period stimulated HEIs to go for online or hybrid solutions, now regulated in Romania.
There is a whole national methodology of evaluating the study programs used by the national body of financing higher education in Romania, not mentioned by the authors. There there are used more differentiated criteria to evaluate the efficiency of the study programs.
In addition, at the beginning of the design part, there were listed statements about HE in Romania, and it is not clear on what they are based, suggesting a certain bias of the researchers: "Moreover, they hardly meet the varied needs of the labor market. Consequently, the educational programs should become more flexible and better-tailored to the demands of the labor market. (line 314) The completion of the study programs, especially those for undergraduate (315) programs, should lead to gaining competences, acquiring specialization knowledge, and developing other educational skills. It should also encourage students to experience different situations, to choose from various options and alternative methods and to take responsibility for their choices. In addition, the educational programs should stimulate and increase the degree of inter- and multi-disciplinarity, as well as soar the amount of (320) acquired information and raise the students’ awareness of sustainability". There are some empirical data to sustain such points of view?
Even the instruments are useful validated tools to be used, it is advisable to make clear the options for the criteria considered in the VRS-DEA and CRS-DEA model of analysis, for the reliability of the data. The criteria used by the national financing body of HEIs are to be considered.
Author Response
Thank you for the useful remarks. We have included modifications to your requests in the revised paper. Below there are punctual answers to your observations.
The article propose a helpful approach for the university management, interested in reducing risks. It is used a reliable model of evaluating how efficient study programs can be while using various resources, so university management can compare their different study programs, within the same university or nation-wide.
However, there are some points of inquiry about the research design used, and aspects considered as inputs and outputs, as it was not explained why they are the most relevant ones, and there were used only them. Why, for instance, square meters are so relevant, mainly the pandemic period stimulated HEIs to go for online or hybrid solutions, now regulated in Romania.
Answer: Data in this paper are from periods before pandemic. However, in the institution we analyzed in the paper, after pandemic the Senate decided to stop any academic activity in the online format, considering the teaching activity is not of good quality.
Referring to the selection of inputs and outputs, explanations have been added in the paragraph above Table 2.
There is a whole national methodology of evaluating the study programs used by the national body of financing higher education in Romania, not mentioned by the authors. There there are used more differentiated criteria to evaluate the efficiency of the study programs.
Answer: Consideration of criteria used by national bodies to allocate funds in higher education in Romania is out of the scope of this research. No remarks on this issue is considered in the paper, because our focus here is on comparing study programs within the university, not comparing HEIs or similar study programs from different universities.
In addition, at the beginning of the design part, there were listed statements about HE in Romania, and it is not clear on what they are based, suggesting a certain bias of the researchers: "Moreover, they hardly meet the varied needs of the labor market. Consequently, the educational programs should become more flexible and better-tailored to the demands of the labor market. (line 314) The completion of the study programs, especially those for undergraduate (315) programs, should lead to gaining competences, acquiring specialization knowledge, and developing other educational skills. It should also encourage students to experience different situations, to choose from various options and alternative methods and to take responsibility for their choices. In addition, the educational programs should stimulate and increase the degree of inter- and multi-disciplinarity, as well as soar the amount of (320) acquired information and raise the students’ awareness of sustainability". There are some empirical data to sustain such points of view?
Answer: There is no counted empirical study to be referenced. However, this is a countless opinion highlighted in many public interventions, as well as on web posts. We consider useful to include a paragraph on these aspects to indicate the relevance of measuring efficiency of the study programs.
Even the instruments are useful validated tools to be used, it is advisable to make clear the options for the criteria considered in the VRS-DEA and CRS-DEA model of analysis, for the reliability of the data. The criteria used by the national financing body of HEIs are to be considered.
Answer: Criteria used by the national financing body will be considered in future research, as it is mentioned in the ~Conclusions~. However, not all criteria can be considered, as long as not all of them are related to the study programs. We have actually taken into account in this research only the number of students (embedded in the number of conventional hours), which is related to the financing (input) of the study program. On the other side, not all criteria used by national bodies reflect efficiency. Many of them are about prestige and other dimensions. We also consider that most of the criteria used by the national bodies must be the subject of a separate research in relation to what is called the Goodhart law, because many of the current used criteria are not correlated to the study programs and to the real needs of the national economy.
Reviewer 3 Report
1. Dear Authors, I would suggest to see the following article:
An application of DEA methodology in efficiency of the czech public universities, by Pavla Mikusova,
2. at line 240: DMUs TO identifY the most efficient units that could form an efficient frontier.
3. At line 255 you present the various models since 1978 (including SRC, VRS), but at line 266 you claim that all DEA models are SRC or VRS. I consider it to be vague.
Author Response
Thank you for the useful remarks. We have included modifications to your requests in the revised paper. Below there are punctual answers to your observations.
- Dear Authors, I would suggest to see the following article:
An application of DEA methodology in efficiency of the czech public universities, by Pavla Mikusova,
Answer: This reference has been included in the paper and commented.
- at line 240: DMUs TO identifY the most efficient units that could forman efficient frontier.
Answer: This issue was fixed
- At line 255 you present the various models since 1978 (including SRC, VRS), but at line 266 you claim that all DEA models are SRC or VRS. I consider it to be vague.
Answer: Corrections have been done. The claim ~all~ was removed.
