Zero Waste Scientific Research Evaluation: The Scientific Research Evaluation System Framework to Stimulate Scholars’ Empathy and Innovation Intention

Round 1

Reviewer 1 Report

Thank you for the opportunity to read this paper. The article is fitting with the journal aim, but I have some suggestion to improve the paper.

The abstract is a little bit to long.

Pleas add to the end of the introduction the novelty and originality of the paper.

I think the literature review must be extended and can be a separate part.

Research Design and Analysis must be connected to the existing literature.

Figure and table must be added in the text. And the discussions must be connected to the existing literature and highlighted the personal contributions.

Some additional literature:

Fülöp, M. T., Breaz, T. O., He, X., Ionescu, C. A., Cordoş, G. S., & Stanescu, S. G. (2022). The role of universities' sustainability, teachers' wellbeing, and attitudes toward e-learning during COVID-19. Frontiers in Public Health, 10.

Aithal, P. S., & Maiya, A. K. (2022). Holistic Integrated Student Development Model & Service Delivery Model–A Best Practice of Srinivas University, India. International Journal of Case Studies in Business, IT, and Education (IJCSBE), 6(1), 590-616.

Aguirre, A., Carballo, R., & Lopez-Gavira, R. (2021). Improving the academic experience of students with disabilities in higher education: faculty members of Social Sciences and Law speak out. Innovation: The European Journal of Social Science Research, 34(3), 305-320.

 

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Reviewer 2 Report

This is interesting research that focuses on developing a research evaluation system framework to stimulate Chinese university scholars' empathy and innovation intention. This study seems comprehensive, and I would like to commend the authors for undertaking this study. However, I have noticed some problems or concerns with the paper.

Figures 2 and 3 appear in the paper, but they are not specifically described or discussed in the paper. Authors should not expect readers to do the hard work trying to figure out the figures. Furthermore, it would be helpful for readers if the authors used some descriptive names for constructs or items on the figure. Using acronyms is okay, but items and variables' descriptions should be provided in a table format.

Did the authors mean "second-order" in Figure 2 and Figure 3 titles? Also, it would be better if the authors differentiated the Figure 2 and Figure 3 titles by providing some other titles.

Readability is poor, particularly in describing the research design. Since this is a large-scale study, I think it would be helpful if a diagram were used to illustrate the research design.

The research discussion and conclusion should be presented in the context of China. 

Some references are cited but do not appear in the reference list (e.g., Davis (1996)). Also, some references do not use a conventional citation style (e.g., Brett (2022) should be Brett et al. (2022))

 

Author Response

Response to reviewer #2

Thank you for the kind comment. We appreciate the time and effort that you put into reviewing our manuscript and providing us with valuable comments and suggestions. We have now made substantial revisions to the manuscript following your suggestions and the manuscript has been significantly improved. All changes we made in this version of the manuscript are highlighted. Your comments are reported below and our responses follow immediately.

• Figures 2 and 3 appear in the paper, but they are not specifically described or discussed in the paper. Authors should not expect readers to do the hard work trying to figure out the figures. Furthermore, it would be helpful for readers if the authors used some descriptive names for constructs or items on the figure. Using acronyms is okay, but items and variables' descriptions should be provided in a table format.

Thanks to your comments. According to your advice, a specific interpretation of the two second-order plots has been provided. The measurement model fitting indexes have been elaborated, and the weight of the second-order formative factors has been elaborated that the first-order factors interpreting the corresponding second-order factors to a higher degree. In addition, we have added a table of variable abbreviations to illustrate the variables represented by the abbreviations. （Please see Table 10）

• Did the authors mean "second-order" in Figure 2 and Figure 3 titles? Also, it would be better if the authors differentiated the Figure 2 and Figure 3 titles by providing some other titles.

Thanks to your comments. According to your advice, we replaced "second second-order model" with "re-validation of the second-order model".

• Readability is poor, particularly in describing the research design. Since this is a large-scale study, I think it would be helpful if a diagram were used to illustrate the research design.

Thanks to your comments. According to your advice, a research design diagram has been added to this section.（Please see Figure 2）

• The research discussion and conclusion should be presented in the context of China.

Thanks to your comments. According to your advice, a description of the Chinese context has been added to the discussion and conclusion of this study. Current research context in China is added at the conclusion, “At present, China's scientific research evaluation system shows the phenomenon of "Five Only", that is, scholars only care about publishing papers, only care about getting honors, only care about titles obtaining, only care about their personal academic improvement and only care about awards winning. The reform measures related to breaking the" Five Only" university scientific research evaluation system have been taken to actively encourage the university scientific research evaluation system to pay more attention to quality and process in order to achieve the purpose of promoting scientific research innovation.” （Please see 8.1 Conclusions ）

• Some references are cited but do not appear in the reference list (e.g., Davis (1996)). Also, some references do not use a conventional citation style (e.g., Brett (2022) should be Brett et al. (2022))

Thanks to your comments. According to your advice, references have been added and the citation format has been corrected.

Author Response File: Author Response.pdf

Reviewer 3 Report

The topic is interesting and has wider practical applications for developing nations. The authors have put their best efforts to execute this paper. However, I have the following reservations and suggestions for the sake of improvement of the undertaken study:

Abstract

The logical sequence of the abstract should be as 1) objectives, 2) methodology, 3) Findings, 4) conclusion and 5) implications. Thus, the authors should also rewrite the abstract in this sequence. The authors should mention the statistical techniques in the methodology, such as the SEM-based approach. In the methodology of an abstract, more emphasis should be given to scaling, questionnaire, and estimation techniques. After the findings conclude and then please describe important implications. 

Introduction

The authors did not establish the motivation, significance, and novelty of the undertaken study. The authors suggested improving this important factor in the "Introduction" section. The authors have presented the citations in parentheses i.e., (Xue et al., 2021), however, it should be written as [1].

Hypotheses & review of literature

The concept definition and hypotheses section is written in a comprehensive manner. I will only recommend drawing a conceptual framework at the end of the hypotheses. The citations are up to date, and quite relevant. 

 

Research design and analysis

The research design and analysis are written in a professional manner; however, it is recommended to the authors to explain more the statistical modeling i.e., the SEM-Based multivariate approach and its ingredients. It is also suggested to the authors to describe the origin of items or scaling of the questionnaire i.e. adapted or adopted. Another sub-heading should be measurement scaling and lastly sub-heading of sampling strategy and sample size.

Results

Since the authors used structural equation modeling using M-Plus software. The pre-requisite sequence of the SEM-based multivariate approach is to first validate the measurement model and then validate the structural model. Thus, we have to use different techniques to satisfy these two conditions. The authors should mention they used FL, CA, CR & AVE to satisfy convergent and discriminant validities and reliabilities, which are important to validate the measurement model. Moreover, Fornell-Larcker Criterion, HTMT, Cross Loading, and R-squared values are also important to validate the measurement model. Secondly, direct and indirect hypothesized relationships or hypotheses testing proved the structural model. Lastly, model fitness should be reflected through SRMR, and predictive relevance values are important. The authors should present the missing values and sequence the sub-headings as mentioned above. It is also requested to the authors place the tables in the original position just after the interpretations of the results.

Discussions & conclusions

The discussions section should be added after the results section. The discussions section provides the opportunity for the authors to sell their idea to the readers. The discussions section should be complemented with the previous literature. The authors should compare their results with the previous literature on whether they are consistent and why? If they are not consistent with the previous literature then what is the reason and how it should be contributed to the body of knowledge? The conclusion should not be a repetition of results, therefore, it needs further elaboration. The conclusion is always one step ahead of the findings. 

Author Response

Response to reviewer #3

Thank you for the kind comment. We appreciate the time and effort that you put into reviewing our manuscript and providing us with valuable comments and suggestions. We have now made substantial revisions to the manuscript following your suggestions and the manuscript has been significantly improved. All changes we made in this version of the manuscript are highlighted. Your comments are reported below and our responses follow immediately.

• Abstract

The logical sequence of the abstract should be as 1) objectives, 2) methodology, 3) Findings, 4) conclusion and 5) implications. Thus, the authors should also rewrite the abstract in this sequence. The authors should mention the statistical techniques in the methodology, such as the SEM-based approach. In the methodology of an abstract, more emphasis should be given to scaling, questionnaire, and estimation techniques. After the findings conclude and then please describe important implications.

Thanks to your comments. According to your advice, we have modified the abstract section in the logical order you described; we have also added an elaboration of the statistical techniques that were employed “Based on grounded theory, this study analyzed the personal information and interview data of 26 university scholars publicly available online as well as by using Nvivo 12 software to confirm the two dimensions of the university research system, based on which the initial scale was developed, and the SEM model was used to verify the scale reliability and validity through Mplus 8.0 software, and to test the influence of the evaluation system on scholars' empathy and willingness to innovate.” （Please see page 2）

 

• Introduction

The authors did not establish the motivation, significance, and novelty of the undertaken study. The authors suggested improving this important factor in the "Introduction" section. The authors have presented the citations in parentheses i.e., (Xue et al., 2021), however, it should be written as [1].

Thanks to your comments. According to your advice, we added a comparison with previous studies in the introduction to emphasize the novelty and significance of this study，“In general, the previous research evaluation system is mostly from the perspective of managers, and the formulation process is easily influenced by subjective ideas, and the evaluation indicators are mostly formulated for the quantity (Kaklauskas et al., 2018) and quality ((Shuqin et al., 2019)) of research results, neglecting the evaluation of the research process and neglecting the influence of environmental factors on university scholars' scientific research; furthermore, previous studies have not developed relevant scales for the evaluation system and have not explored the relationship between the evaluation system and scholars' empathy and willingness to innovate. In contrast, this study introduces the perspective of university scholars and refines the indicators of university research evaluation system through substantive analysis and deeply integrates the quantity of results, quality, environment and research process. On this basis, a mixed research method combining qualitative and quantitative approaches is used for scale development, which comprehensively reveals the influence of university research evaluation system on scholars' empathy and willingness to innovate.” （Please see page 4）

In addition, the references of this study are in alphabetical order according to the authors' names, so this literature corresponds to the fifty-first.

• Hypotheses & review of literature

The concept definition and hypotheses section is written in a comprehensive manner. I will only recommend drawing a conceptual framework at the end of the hypotheses. The citations are up to date, and quite relevant.

Thanks to your comments. According to your advice, we have moved the figure after the study hypothesis.

• Research design and analysis

The research design and analysis are written in a professional manner; however, it is recommended to the authors to explain more the statistical modeling i.e., the SEM-Based multivariate approach and its ingredients. It is also suggested to the authors to describe the origin of items or scaling of the questionnaire i.e. adapted or adopted. Another sub-heading should be measurement scaling and lastly sub-heading of sampling strategy and sample size.

Thanks to your comments. According to your advice, we have added a specific description of the model component, “The independent variables of the research model were adopted from the university research evaluation system scale developed in this study, which consists of 25 items; the scholars' empathy scale was adopted from the ACME empathy scale (Brett et al., 2022), and two dimensions of cognitive empathy and emotional empathy were selected, with 14 items, including 6 items of cognitive empathy and 8 items of emotional empathy; the willingness to innovate scale was adopted from the Innovation Scale (Cheng, 2008), with a total of 8 items. The questionnaire was designed using a 5-point Likert scale, with "1" indicating "strongly disagree" and "5" indicating "strongly agree"”. （Please see page 26）

Additional subheadings on measurement proportions, sampling strategy, and sample size have been added as well.

• Results

Since the authors used structural equation modeling using M-Plus software. The pre-requisite sequence of the SEM-based multivariate approach is to first validate the measurement model and then validate the structural model. Thus, we have to use different techniques to satisfy these two conditions. The authors should mention they used FL, CA, CR & AVE to satisfy convergent and discriminant validities and reliabilities, which are important to validate the measurement model. Moreover, Fornell-Larcker Criterion, HTMT, Cross Loading, and R-squared values are also important to validate the measurement model. Secondly, direct and indirect hypothesized relationships or hypotheses testing proved the structural model. Lastly, model fitness should be reflected through SRMR, and predictive relevance values are important. The authors should present the missing values and sequence the sub-headings as mentioned above. It is also requested to the authors place the tables in the original position just after the interpretations of the results.

Thanks to your comments. According to your advice, we went searching for literature and books on Mplus structural equation modeling and added the followings, “The absolute fit indicators of the second-order model all met the criteria, with the absolute fit index χ²/df of 1.784 and RMSEA of 0.072, both less than 0.08; CFI and TLI of 0.923 and 0.937, respectively, both greater than 0.9; although the SRMR of 0.084 was greater than 0.08, it was still within the acceptable range, indicating that the overall fitness of the model was good.” （Please see page 28）

Add the following reference：

Baumgartner, H., & Homburg, C. (1996). Applications of structural equation modeling in marketing and consumer research: A review. International journal of Research in Marketing, 13(2), 139-161.

• Discussions & conclusions

The discussions section should be added after the results section. The discussions section provides the opportunity for the authors to sell their idea to the readers. The discussions section should be complemented with the previous literature. The authors should compare their results with the previous literature on whether they are consistent and why? If they are not consistent with the previous literature then what is the reason and how it should be contributed to the body of knowledge? The conclusion should not be a repetition of results, therefore, it needs further elaboration. The conclusion is always one step ahead of the findings. 

Thanks to your comments. According to your advice, we place the discussion of each chapter after the conclusion and describe the novelty of this study in comparison with previous literature and a further explanation has also been made in the conclusion section, “This study holds that the scientific research process can be divided into three stages, namely, project initiation, implementation and acceptance. In the process of project initiation, the higher the expected economic and social benefits or benefits brought by the scientific research project (vertical) by the university, the more scholars will spend on refining their own projects, and the high-quality projects need to be based on innovative thinking; In the implementation stage and the acceptance stage, university scholars are given more time to produce high-quality works. Additionally, the award of scientific research achievements will promote scholars' recognition of the university scientific research system.” （Please see page 31）

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

Thank you for improving the paper.

Congratulations for a new publication!

 

 

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Reviewer 2 Report

Thank you for your efforts. You have made significant improvements since the last version. In my opinion, this paper deserves to be published in an academic journal. 

Author Response

October 20th, 2022

 

Dear professor:

 

I wish to submit an original article for consideration of publication at sustainability, titled “Zero Waste Scientific Research Evaluation: The Scientific Research Evaluation System to Stimulate University Scholars’Empathy and Innovation Intention”. This paper was coauthored by Kui Yi, Pingping Li and He Yu. He Yu was added as a correspondent for the following reasons: (1)In this revision, he played a great role. For example, the language of the whole paper was revised again, and the references were further checked. (2)In the early writing, this person participated in the whole process of paper writing, and gave suggestions for revision. So we added him as a correspondent. In this revision, He  Yu played a major role, and at the same time, he participated in the writing of the whole paper. I hope the editor can understand it , and I would like to pay my greatest tribute to the editor here.

 

This study conducted three sub-studies to develop a comprehensive and scientific measurement scale and verify the impact of the university research evaluation system on scholars' empathy and willingness to innovate through the scale. We analyzed the personal information and interview data of 26 university scholars publicly available online to confirm the two dimensions of the university research system, based on which the initial scale was developed.The SEM model was used to verify the scale reliability and validity, and to test the influence of the evaluation system on scholars' empathy and willingness to innovate. This study expects to fully stimulate scholars' empathy and Innovation Intention, and reduce unnecessary educational and political investment, which is important for the sustainable development of society. 

 

For the editors’ and referees’ approval, we have made extensive revisions to our paper.In the abstract, we further refine the content and logically sort it out in the order of 1) objectives, 2) methodology, 3) findings, 4) conclusion and 5) implications. In the introduction, we added the novelty and innovation of the paper in the last paragraph. In the review part, we have adjusted the review structure, divided the literature review part into one section, and changed the original review framework of "scholars' empathy" and "innovation intention" into "emotional feedback of scientific research evaluation system", "empathy, willingness and zero waste", so as to enhance the relationship between reviews and the depth of reviews, and to supplement the content and broaden the width of reviews. In the part of research design and analysis, we added the research design drawing, contacted the existing literature, elaborated the elements of the model in detail, and added the subheadings of measurement ratio. In the third sub-study, we added the overall model fitting index to meet the statistical standard. In addition, we have adjusted the order of papers and charts to meet the format requirements of journals.

 

In this revision process, we have revised the language of the whole paper, adding 8.2 discussion section, and in this section, we added 8.2.4 general discussion to expand the research content and significance of this paper. And we checked again the reference part, added references more relevant to the research topic.

 

We confirm that this paper represents original work and is not under review elsewhere.

 

Thank you very much for your time and consideration!  

 

Sincerely,

 

Kui Yi

School of Economics and Management, East China Jiaotong University, Nanchang, China,

Email: [email protected]

 

 

 

Funding information: This study is supported by Fund Project of Jiangxi Natural Science Foundation Project (No. 20212BAA10009).

Author Response File: Author Response.docx