Administrative Processes Efficiency Measurement in Higher Education Institutions: A Scoping Review

Round 1

Reviewer 1 Report

Title: “Administrative Processes Efficiency Measurement in Higher Education Institutions: A Scoping Review”

 

The article is prepared on a current topic; the article is of interest to scientists; and the article can be published, but it needs preliminary thorough revision. The article is devoted to the identifying of the main methods of efficiency evaluation in the administrative processes of HEIs, as well as indicators associated with these models.

 

Suggestions for authors:

1) I think, it is advisable to clearly identify the hypotheses of the study; and in the section Conclusions to determine how these  hypotheses were proven within the study;

2) in our opinion, it is appropriate to separate the section Conclusion, and summarize research results within this section;

3) I think, the authors should pay more attention to the description of the relevance of the research topic;

4) in my opinion, at the end of the article, it is worth describing the prospects for further research on the subject of the article

5) I think, the Abstract should be expanded by describing in more detail the results of the research and its limitations.

Author Response

 

Author Response File: Author Response.pdf

Reviewer 2 Report

The research question on page 2, line 67 is wrongly formulated: it should say how instead of who.

The authors devote a lot of space to discussing how they collect the evidence, the objective, the perimeter, and the selection of papers. All the former information is contained in Figure 1 and can be complemented with a page of detail, no more than that.

Figures 2 and 3 are completely irrelevant.

Subsection 3.2.1 and Section 4 are the core of the study.

The study lacks a contextual discussion on efficiency, quality, and perception of quality. It seems that the authors use interchangeably the concepts of quality (understood as objective and measurable attributes) and perception of quality (understood as subjective and measurable by ordinal scales in a survey, for instance, which is conventionally cardinalized). If they do not distinguish quality from quality perception, they are summing app apples and pears. Administrative efficiency can be computed in some objective way, such as the time students spend in the administrative processes, time to respond to queries, part of the fees they pay which is spent in administrative processes, and the like, which is conceptually different from the degree of satisfaction expressed in a survey.

The authors can improve their research by better understanding the issue they are studying. What is quality? What is administrative efficiency? How it is measured and why? How can it be improved? Efficiency is a matter of attaining objectives at the least cost in resources (money, effort, time, whatever the variables you chose for metrics).

The citation method of the journal uses [number] in the text instead of Author(date). The authors mix in several parts of the paper the citation methods, for example in page 1 Agasisti[12]. Please, control and correct it.

On page 4, line 135 there is a reference to "key aspects of efficiency in administrative process, such XXX(sic)". Please, complete.

English is correct but can be improved. One easy and economic way is using some software such as Grammarly. A more thorough and probably expensive way is making it reviewed by a native speaking person.

Author Response

Response to Reviewer 2 Comments

Point 1: The research question on page 2, line 67 is wrongly formulated: it should say how instead of who.

Response 1: The research question on page 2 was reformulated as shown below:

“The research question posed in this scoping review is how has the efficiency of administrative processes in Higher Education Institutions (HEIs) been measured?”

 

Point 2: The authors devote a lot of space to discussing how they collect the evidence, the objective, the perimeter, and the selection of papers. All the former information is contained in Figure 1 and can be complemented with a page of detail, no more than that.

Figures 2 and 3 are completely irrelevant.

Subsection 3.2.1 and Section 4 are the core of the study.

 

Response 2: Because the discussion was conducted in accordance with a reference guide, it is considered relevant to the article's scope as a way to ensure that it could be replicated. But, figures 2 and 3 were eliminated as suggested. Additionally, the methodology was registered in an Open Framework as suggested by the journal.

 

Point 3: The study lacks a contextual discussion on efficiency, quality, and perception of quality. It seems that the authors use interchangeably the concepts of quality (understood as objective and measurable attributes) and perception of quality (understood as subjective and measurable by ordinal scales in a survey, for instance, which is conventionally cardinalized). If they do not distinguish quality from quality perception, they are summing apples and pears. Administrative efficiency can be computed in some objective way, such as the time students spend in the administrative processes, time to respond to queries, part of the fees they pay which is spent in administrative processes, and the like, which is conceptually different from the degree of satisfaction expressed in a survey.

The authors can improve their research by better understanding the issue they are studying. What is quality? What is administrative efficiency? How it is measured and why? How can it be improved? Efficiency is a matter of attaining objectives at the least cost in resources (money, effort, time, whatever the variables you chose for metrics).

 

Response 3: We rewrite and add paragraphs embodying the idea it tells us. This can be reviewed in the section (1) introduction and section (5) conclusions.

 

Point 4: The citation method of the journal uses [number] in the text instead of Author(date). The authors mix in several parts of the paper the citation methods, for example in page 1 Agasisti [12]. Please, control and correct it.

 

Response 4: The citation method has been corrected.

 

Point 5: On page 4, line 135 there is a reference to "key aspects of efficiency in administrative process, such XXX(sic)". Please, complete.

 

Response 5: The sentence has been completed.

 

Author Response File: Author Response.pdf

Reviewer 3 Report

The paper shows promise; however, it would be even more engaging if the authors expanded beyond efficiency measurement. Currently, there is a growing research trend focused on the application of Lean Thinking in Higher Education Institutions, which the authors appear to overlook. It is essential to acknowledge that lean application is closely tied to efficiency. By disregarding this aspect, the authors are missing out on a substantial body of relevant literature.

Furthermore, there are certain decisions made by the authors that lack justification. For instance, why did they specifically search for Spanish papers? And, why 2017? Why Scopus? They have included numerous publications in Spanish language, but it would have been preferable if they had at least provided the English translation of the titles.

Overall, addressing these points and considering a broader scope of research would greatly enhance the paper's impact and relevance.

It seems ok but I´m not a native English speaker.

 

Author Response

.

Author Response File: Author Response.pdf

Round 2

Reviewer 2 Report

The author(s) followed the recommendations made by this referee

Author Response

We were notified that Reviewer 2 was satisfied with the previous submissions' observations.
Thank you very much.

Reviewer 3 Report

The paper has a small improvement and continues needing many improvements. By not including the word “Lean” the authors missing many updated important papers like the examples provided:

1.       Antony, J., Ghadge, A., Ashby, S. A., & Cudney, E. A. (2018). Lean Six Sigma journey in a UK higher education institute: a case study. International Journal of Quality & Reliability Management, 35(2), 510–526. https://doi.org/10.1108/IJQRM-01-2017-0005

2.       Balzer, W. K., Brodke, M. H., & Thomas Kizhakethalackal, E. (2015). Lean higher education: successes, challenges, and realizing potential. International Journal of Quality & Reliability Management, 32(9), 924–933. https://doi.org/10.1108/IJQRM-08-2014-0119

3.       Douglas, J. A., Antony, J., Ciasullo, M. V., & Douglas, A. (2020). Recognising Waste in Higher Education Institutions Using Lean Thinking. In Lean Six Sigma for Higher Education (pp. 21–38). WORLD SCIENTIFIC (EUROPE). https://doi.org/10.1142/9781786348500_0002

4.       Douglas, J., Antony, J., & Douglas, A. (2015). Waste identification and elimination in HEIs: the role of Lean thinking. International Journal of Quality & Reliability Management, 32(9), 970–981. https://doi.org/10.1108/IJQRM-10-2014-0160

5.       Klein, L.L., Alves, A. C., Abreu, M. F., & Feltrin, T. S. (2022). Lean management and sustainable practices in Higher Education Institutions of Brazil and Portugal: A cross country perspective. Journal of Cleaner Production, 342. https://doi.org/10.1016/j.jclepro.2022.130868

6.       Klein, L.L., De Guimarães, J. C. F., Severo, E. A., Dorion, E. C. H., & Schirmer Feltrin, T. (2023). Lean practices toward a balanced sustainability in higher education institutions: a Brazilian experience. International Journal of Sustainability in Higher Education, 24(2), 259–278. https://doi.org/10.1108/IJSHE-10-2020-0406

7.       Klein, L.L., Tonetto, M. S., Avila, L. V., & Moreira, R. (2021). Management of lean waste in a public higher education institution. Journal of Cleaner Production, 286. https://doi.org/10.1016/j.jclepro.2020.125386

8.       Klein, Leander Luiz, Alves, A. C., Abreu, M. F., & Feltrin, T. S. (2022). Lean management and sustainable practices in Higher Education Institutions of Brazil and Portugal: A cross country perspective. Journal of Cleaner Production, 342, 130868. https://doi.org/10.1016/j.jclepro.2022.130868

9.       Klein, Leander Luiz, De Guimarães, J. C. F., Severo, E. A., Dorion, E. C. H., & Schirmer Feltrin, T. (2021). Lean practices toward a balanced sustainability in higher education institutions: A Brazilian experience. International Journal of Sustainability in Higher Education. https://doi.org/10.1108/IJSHE-10-2020-0406

10.   Klein, Leander Luiz, Vieira, K. M., Alves, A. C., & Pissutti, M. (2023). Demystifying the eighth lean waste: a knowledge waste scale. International Journal of Quality & Reliability Management. https://doi.org/10.1108/IJQRM-01-2022-0020

11.   Klein, Leander Luiz, Vieira, K. M., Marçal, D. R., & Pereira, J. R. L. (2022). Lean management practices perception and their influence on organizational performance in a public Higher Education Institution. TQM Journal. https://doi.org/10.1108/TQM-11-2021-0311

12.   O’Reilly, S. J. S. J., Healy, J., Murphy, T., & Ó’Dubhghaill, R. (2019). Lean Six Sigma in higher education institutes: an Irish case study. International Journal of Lean Six Sigma, 10(4), 948–974. https://doi.org/10.1108/IJLSS-08-2018-0088

 

Furthermore, the paper continue without answers to the questions that I raised in the last revision, namely: why a search for Spanish papers? Why 2017? Why Scopus? The authors have included numerous publications in Spanish language, but it would have been preferable if they had at least provided the English translation of the titles. The authors should update the review to include publications of 2023, otherwise the study will be out of date. 

Needs some improvements.

 

Author Response

Response to Reviewer 3 Comments

Point 1: The paper has a small improvement and continues needing many improvements. By not including the word “Lean” the authors missing many updated importan papers like the examples provided:

1. Antony, J., Ghadge, A., Ashby, S. A., & Cudney, E. A. (2018). Lean Six Sigma journey in a UK higher education institute: a case study. International Journal of Quality & Reliability Management, 35(2), 510–526. https://doi.org/10.1108/IJQRM-01-2017-0005
2. Balzer, W. K., Brodke, M. H., & Thomas Kizhakethalackal, E. (2015). Lean higher education: successes, challenges, and realizing potential. International Journal of Quality & Reliability Management, 32(9), 924–933. https://doi.org/10.1108/IJQRM-08-2014-0119
3. Douglas, J. A., Antony, J., Ciasullo, M. V., & Douglas, A. (2020). Recognising Waste in Higher Education Institutions Using Lean Thinking. In Lean Six Sigma for Higher Education (pp. 21–38). WORLD SCIENTIFIC (EUROPE). https://doi.org/10.1142/9781786348500_0002
4. Douglas, J., Antony, J., & Douglas, A. (2015). Waste identification and elimination in HEIs: the role of Lean thinking. International Journal of Quality & Reliability Management, 32(9), 970–981. https://doi.org/10.1108/IJQRM-10-2014-0160
5. Klein, L.L., Alves, A. C., Abreu, M. F., & Feltrin, T. S. (2022). Lean management and sustainable practices in Higher Education Institutions of Brazil and Portugal: A cross country perspective. Journal of Cleaner Production, 342. https://doi.org/10.1016/j.jclepro.2022.130868
6. Klein, L.L., De Guimarães, J. C. F., Severo, E. A., Dorion, E. C. H., & Schirmer Feltrin, T. (2023). Lean practices toward a balanced sustainability in higher education institutions: a Brazilian experience. International Journal of Sustainability in Higher Education, 24(2), 259–278. https://doi.org/10.1108/IJSHE-10-2020-0406
7. Klein, L.L., Tonetto, M. S., Avila, L. V., & Moreira, R. (2021). Management of lean waste in a public higher education institution. Journal of Cleaner Production, 286. https://doi.org/10.1016/j.jclepro.2020.125386
8. Klein, Leander Luiz, Alves, A. C., Abreu, M. F., & Feltrin, T. S. (2022). Lean management and sustainable practices in Higher Education Institutions of Brazil and Portugal: A cross country perspective. Journal of Cleaner Production, 342, 130868. https://doi.org/10.1016/j.jclepro.2022.130868
9. Klein, Leander Luiz, De Guimarães, J. C. F., Severo, E. A., Dorion, E. C. H., & Schirmer Feltrin, T. (2021). Lean practices toward a balanced sustainability in higher education institutions: A Brazilian experience. International Journal of Sustainability in Higher Education. https://doi.org/10.1108/IJSHE-10-2020-0406
10. Klein, Leander Luiz, Vieira, K. M., Alves, A. C., & Pissutti, M. (2023). Demystifying the eighth lean waste: a knowledge waste scale. International Journal of Quality & Reliability Management. https://doi.org/10.1108/IJQRM-01-2022-0020
11. Klein, Leander Luiz, Vieira, K. M., Marçal, D. R., & Pereira, J. R. L. (2022). Lean management practices perception and their influence on organizational performance in a public Higher Education Institution. TQM Journal. https://doi.org/10.1108/TQM-11-2021-0311
12. O’Reilly, S. J. S. J., Healy, J., Murphy, T., & Ó’Dubhghaill, R. (2019). Lean Six Sigma in higher education institutes: an Irish case study. International Journal of Lean Six Sigma, 10(4), 948–974. https://doi.org/10.1108/IJLSS-08-2018-0088

As you mentioned, the lean philosophy application is closely tied to efficiency. As a result of this study, 30% of the articles used the lean philosophy in some way. It is worth mentioning that from the relevant literature that you sent us, article 12 is part of the studies selected in our study that met the eligibility criteria.

Following your recommendations, we are highlighting lean topics in several sections: 1 Introduction, 4 in the discussion of the articles' results, and in section 5, as a trend for future research. Also, the abstract section was updated.

Additionally, we add recent bibliography on the lean philosophy that is shown below:

1. Mayanga, C.S; Participatory monitoring and evaluation for quality programs in higher education: What is the way for Uganda?. Int. J. Educ. Adm. Policy Stud. 2020, 12, 52–59.

3. Texeira-Quiros, J., Justino, M. do R., Antunes, M.G., Mucharreira, P.R., Nunes, A. de T. Effects of Innovation, Total Quality Management, and Internationalization on Organizational Performance of Higher Education Institutions. Frontiers in Psychology. 2022, 13(April), 1–11. https://doi.org/10.3389/fpsyg.2022.869638

5. He, L., Ismail, K. Do staff capacity and performance-based budgeting improve organisational performance? Empirical evidence from Chinese public universities. Humanities and Social Sciences Communications. 2023, 10(1). https://doi.org/10.1057/s41599-023-01523-2

6. Matalka, M.Al, Zoubi, M.Al. The influence of soft and hard quality management practices on quality improvement and performance in UAE higher education. International Journal of Data and Network Science. 2023, 7(3), 1311–1320. https://doi.org/10.5267/j.ijdns.2023.4.007

7. Verschueren, N., Van Dessel, J., Verslyppe, A., Schoensetters, Y., Baelmans, M. A Maturity Matrix Model to Strengthen the Quality Cultures in Higher Education. Education Sciences. 2023, 13(2), 123. https://doi.org/10.3390/educsci13020123

8. Salinas-Navarro, D. E., Mejia-Argueta, C., Montesinos, L., Rodriguez-Calvo, E. Z. Experiential Learning for Sustainability in Supply Chain Management Education. Sustainability (Switzerland). 2022, 14(20), 1–31. https://doi.org/10.3390/su142013133

9. Tran, C.D.T.T., Battese, G. E., Villano, R. A. Administrative capacity assessment in higher education: The case of universities in Vietnam. International Journal of Educational Development, 77. 2020. https://doi.org/10.1016/j.ijedudev.2020.102198

50. Antony, J.; Ghadge, A.; Ashby, S. A.; Cudney, E. A. Lean Six Sigma journey in a UK higher education institute: a case study. International Journal of Quality and Reliability Management. 2018, 35(2), 510–526. https://doi.org/10.1108/IJQRM-01-2017-0005

51. Klein, L. L.; Alves, A. C.; Abreu, M. F.; Feltrin, T. S. Lean management and sustainable practices in Higher Education Institutions of Brazil and Portugal: A cross country perspective. Journal of Cleaner Production. 2022, 342(August 2021), 130868. https://doi.org/10.1016/j.jclepro.2022.130868

52. Carvalho, D.; Souza, E. De.; Ualison, R.; Fernandes, A.; Teodoro, P. Sustainability in Public Universities through lean evaluation and future improvement for administrative processes. 2023, 382(March 2022). https://doi.org/10.1016/j.jclepro.2022.135318

53. Mulyana, I. J.; Singgih, M. L.; Partiwi, S. G.; Hermanto, Y. B. Identification and Prioritization of Lean Waste in Higher Education Institutions (HEI): A Proposed Framework. Education Sciences. 2023, 13(2). https://doi.org/10.3390/educsci13020137

54. Pongboonchai-Empl, T., Antony, J., Garza-Reyes, J. A., Komkowski, T., & Tortorella, G. L. Integration of Industry 4.0 technologies into Lean Six Sigma DMAIC: a systematic review. Production Planning and Control. 2023, 0–26. https://doi.org/10.1080/09537287.2023.2188496

 

Point 2: Furthermore, the paper continue without answers to the question that I raised in the last revision, namely:

why a search for Spanish papers?

In addition to the fact that the authors belong to a Spanish-speaking university, we considered searching for papers in the Spanish language to cover a wider range of publications on the subject.

As a result of the database spanish search, 21 documents were obtained, of which 2 met the eligibility criteria (10%). It is worth mentioning that one of the articles in Spanish discussed the subject of Six Sigma and also.

This search for english and spanish papers consideration was taken in the Conclusions section of the paper as a limitation of the research.

Lastly, it is worth mentioning that all the titles in Spanish were translated according to your recommendation and recent bibliography in english language was included to the review.

Why 2017?

The authors’ team designed the research considering publications from the past 5 years, as a representative number of years.

Also, as mentioned in point 1, recent articles were included as well, in english language from 2020 to 2023.

Why Scopus?

As mentioned in section 2.1 of the review paper, the Scopus database is one of the most extensive bibliometric data sources for high-impact research, as well as provides extensive coverage in the social sciences field object of our investigation.

In addition to the above, data source availability and the authors' past experience with software tools for reference manipulation were considered.

Finally, the fact that the information was extracted from the Scopus database only, was included in the Conclusions section of the review paper. This was a limitation of the research.

References 27 and 28 in the materials and methods section (2.1) support this source of information.

 

Point 3: The authors have included numerous publications in Spanish language, but it would have been preferable if they had at least provided the English translation of the titles.

During the literature update, articles written in Spanish were eliminated. The titles of the 5 papers that remained were translated into the English language as you can see in the references section with the following numbers:

12. de la C. Hernández, F.D.; Vargas, J.A.; Almuiñas, R.J. Importance of measuring academic efficiency at universities. Rev. Cuba. Educ. Super. 2020, 39.

13. López, S.A; Albíter, R.A.; Ramírez, R.L. Terminal efficiency in Higher Education, the need for a new paradigm. Rev. la Educ. Super. 2008, XXXVII(2), 135–151.

19. Briones, C.P.; Molina, O.S.; Avilés, N.M. Evaluation model of information systems applied to the quality of university administrative management. ProSciences. 2020, 4, 69–89.

31. Alvarado, P.L.; Moreno, F.Z. Governance and quality management in university research centers. Opcion. 2017, 33, 471–503.

41. Fontalvo, T. J.; de la Hoz, E.J.; Marrugo, N. Performance evaluation and efficiency analysis of the sigma level in the evaluation of service quality at a higher education institution. Form. Univ., 2020, 13, 247–254. http://dx.doi.org/10.4067/S0718-50062020000600247

The eliminated literature was:

2. González, A.O.; Batista, M.A. Virtualización del proceso de extensión universitaria: una emergencia en tiempos de Covid-19 / Virtualization of the university extensión process: an emergency in times of Covid-19 . Rev. Univ. y Soc. 2021, 13, 213–222.

5. Rivero, A.K.; Galarza, L.J. El cuadro de mando integral como una alternativa para el seguimiento y control de la estrategia en las instituciones de educación superior / The comprehensive chain of command as an alternative for the follow-up and control of the strategy in the higher education institutions. Rev. Cuba. Educ. Super. 2017, 65, 85–90.

6. Gastelum-Acosta, C; Limon-Romero, J; Maciel-Monteon, M; Baez-Lopez, Y. Seis Sigma en Instituciones de Educación Superior en México/ Six Sigma in Higher Education Institutions in Mexico. Inf. Tecnol. 2018, 29, 91–100.

7. López, J.G.; González, H.S., Rivero, J. L. A. Metodología para evaluar la calidad de la formación del estudiante en la escuela latinoamericana de medicina / A methodology for evaluating the quality of student training in the Latin American School of Medicine. Rev. Cuba. Educ. Medica Super. 2017, 31, 1–19.

 

Point 4: The authors should update the review to include publications of 2023, otherwise the study will be out of date.

The article review has been updated, including 12 recent publications from 2020 to 2023, which are included in point 1 of this document.

 

Round 3

Reviewer 3 Report

The authors did a great effort in improving the paper and almost all comments were addressed. Just a few recommendations in order to improve the contents, text and referencing.

Related to the contents, I believe the connection established by the authors between HEI efficiency and lean methodology needs improvement. It is essential to acknowledge that they introduced this link after page 7 when discussing case studies, rather than addressing it in the introduction where it would have been more appropriate. By presenting a more coherent narrative, the authors can strengthen their argument and improve the overall impact of their research.

Related to the referencing and formatting:

·       Page 7 of 16, line 226: the authors referred four references including [30] that is Rayyan Colaborative Web Aplication. Is this intentional or a mistake?

·       Page 11 of 16, the last paragraph is too long, the authors should try to have shorter sentences.

·       The authors introduced the reference [52] as:

·       Carvalho, D.; Souza, E. De.; Ualison, R.; Fernandes, A.; Teodoro, P. Sustainability in Public Universities through lean evaluation and 528 future improvement for administrative processes. 2023, 382(March 2022). https://doi.org/10.1016/j.jclepro.2022.135318

 

but the first author is not Carvalho!

 Page 11 of 16, the last paragraph is too long, the authors should try to have shorter sentences.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf