Scientific Excellence and Publication Patterns: The Winning Applicants of the Bolyai János Research Scholarship in Hungary in 2021

Round 1

Reviewer 1 Report (New Reviewer)

1. I think explaining the abstract would be a good idea. The abstract should include quantitative results of calculations. The tasks set and the results obtained, which are described in the article, should be clearly understood.

2. From the introduction and the second chapter, it should be clear why precisely this topic was chosen for research and how this research correlates with other similar ones. I recommend clarifying these.

3. The Definition of Scientific Excellence is well-written. Also, it would be interesting to consider applicants by gender. At least it would be generalized to give such an assessment. In particular, the authors may consider citing the following paper (https://www.mdpi.com/2304-6775/11/3/37), which investigates an analysis of the citation impact of scientific publications by authors of different gender compositions.

4. I recommend clarifying hypothesis H2. All hypotheses in the study must be proven. The article should clearly define the tasks (in part 2) and answer them clearly (in the conclusions). The conducted experiment must support all conclusions. Conclusions should contain quantitative research results. At the moment, the conclusions still need to be fully substantiated.

5. The data in the article and the results obtained are interesting. However, more detailed visualization of the data is required. Conducting a cluster analysis will allow us to draw more reasonable conclusions.

6. It is necessary to make the text of the article and the references follow the journal's requirements. Please check the reference list. Some references need more elements. The text has errors, so you need to read it carefully. The text has yellow highlights. I need clarification on what they mean. The quality of the drawings must be improved.

English should be improved.

Author Response

Thank you very much for your valuable comments and observations. After a careful consideration, we sought to incorporate the majority of them into ur article in order to give a clearer explanation of its research aims and raise its overall quality.

‘1. I think explaining the abstract would be a good idea. The abstract should include quantitative results of calculations. The tasks set and the results obtained, which are described in the article, should be clearly understood.’

Thank you for your comment, the abstract was rewritten accordingly.

 

‘2. From the introduction and the second chapter, it should be clear why precisely this topic was chosen for research and how this research correlates with other similar ones. I recommend clarifying these.’

Some parts of the Introduction were rewritten in order to make our choice of this topic clearer.

 

‘3. The Definition of Scientific Excellence is well-written. Also, it would be interesting to consider applicants by gender. At least it would be generalized to give such an assessment. In particular, the authors may consider citing the following paper (https://www.mdpi.com/2304-6775/11/3/37), which investigates an analysis of the citation impact of scientific publications by authors of different gender compositions.’

Thank you for your suggestion, it is a valid and important aspect to be examined. The reason why we did not pay more attention to it in our paper is the relatively small sample size available to work with and our focus on the publication habits of the winning applicants. On page 9, we write about the gender composition of the awardees, as it turns out, the proportion of male winners is almost 75 %.

 

‘4. I recommend clarifying hypothesis H2. All hypotheses in the study must be proven. The article should clearly define the tasks (in part 2) and answer them clearly (in the conclusions). The conducted experiment must support all conclusions. Conclusions should contain quantitative research results. At the moment, the conclusions still need to be fully substantiated.’

Thank you for your comment, our second hypothesis was rephrased and clear references to  all three hypotheses were given in the Conclusion section.

 

‘5. The data in the article and the results obtained are interesting. However, more detailed visualization of the data is required. Conducting a cluster analysis will allow us to draw more reasonable conclusions.’

Although we agree with your comment in principle, we decided not to conduct a cluster analysis because the number of elements rendered into each cluster would be too low. Again, it leads us back to the question of the relatively small sample size.

 

‘6. It is necessary to make the text of the article and the references follow the journal's requirements. Please check the reference list. Some references need more elements. The text has errors, so you need to read it carefully. The text has yellow highlights. I need clarification on what they mean. The quality of the drawings must be improved.’

Thank you for you comments. The text has been proofread, the yellow highlights, which had been used to show earlier changes in the article, were removed.

Reviewer 2 Report (New Reviewer)

This is an interesting study analyzing the winning applicants of the Bolyai János Research Scholarship in Hungary in 2021. However, there are several concerns and areas for improvement that need to be addressed:

1. In the abstract, the authors claim that the descriptive statistics presented in the paper can be used as benchmark figures by future applicants aiming for the research scholarship. However, this assumption may be weak due to the specific focus of the study on the Bolyai János Research Scholarship in Hungary. Scholarship programs and their criteria vary across countries and institutions, making it challenging to generalize the findings. Furthermore, criteria for winning applicants may change over time, and publishing preferences can differ based on disciplines and regions.

2. The paper lacks a separate methodology section, which is essential for transparency and replicability. It would be beneficial to clearly outline the methodology used in the study.

3. The data collection procedure should be explicitly described. For example, the paper should explain how the publication habits of the winning applicants were searched and retrieved from Scopus and the Hungarian Scientific Bibliography.

4. The manuscript uses various subject fields of study in different instances, but it is unclear how the authors conducted specific frequency counts given the potential incompatibility between the subject categories of the Hungarian Scientific Bibliography Database and Scopus. The authors should clarify how they conducted subject field analysis and provide more accurate information.

5. The paper references five publishers based on citation [34], but considering the rapidly evolving publishing industry, it would be more appropriate to cite more recent studies. For instance, the authors could reference the study by Kim and Atteraya (2023) titled "A decade of changes in OA and non-OA journal publication and production" published in the Journal of Librarianship and Information Science, volume 0, issue 0, with page numbers. The DOI for the article is: 

https://doi.org/10.1177/09610006231179718

6. The last sentence of the conclusion lacks context and does not clearly explain the researcher replacement training program or its relation to the research findings. The authors should provide a more detailed explanation to improve clarity.

7. The conclusion should synthesize the main findings of the research and offer deeper analysis and implications. Currently, it mainly presents the findings without providing a comprehensive synthesis or discussing the broader implications of the study.

 

Proofreading is also necessary to correct some awkward expressions and vague statements.

Author Response

Thank you very much for your valuable comments and observations. After a careful consideration, we sought to incorporate the majority of them into ur article in order to give a clearer explanation of its research aims and raise its overall quality.

1. ‘In the abstract, the authors claim that the descriptive statistics presented in the paper can be used as benchmark figures by future applicants aiming for the research scholarship. However, this assumption may be weak due to the specific focus of the study on the Bolyai János Research Scholarship in Hungary. Scholarship programs and their criteria vary across countries and institutions, making it challenging to generalize the findings. Furthermore, criteria for winning applicants may change over time, and publishing preferences can differ based on disciplines and regions.’

In line with the comment, some parts of the abstract were rewritten so that the international context of publication patterns and their relationships with the winning applicants of the Bolyai János Research Scholarship could be more interpretable. However, it is important to mention that one of the main findings of our article, that is, the increasing number of open access articles written by the winning applicants aligns with a palpable international trend observed in the practice of international scientific publication, regardless of the changing criteria of scholarship programmes across countries.

 

2. ‘The paper lacks a separate methodology section, which is essential for transparency and replicability. It would be beneficial to clearly outline the methodology used in the study.’

As our article uses descriptive statistical tools, at first we decided to incorporate the methodology in Chapter 2 and 3, respectively. In order to make our paper more readable, we took your advice and wrote a short chapter on methodology to link the chapter on scientific excellence to the actual results of our study more seamlessly.

 

3. The data collection procedure should be explicitly described. For example, the paper should explain how the publication habits of the winning applicants were searched and retrieved from Scopus and the Hungarian Scientific Bibliography.’

Thank you for your comment. The issue mostly relates to methodology, and the process was fairly simple. As it is stated in the short methodology section, the common ground was the type of publication since both databases use the same taxonomy of publication types. Furthermore, the articles appearing in Scopus-indexed journals are automatically shown in the Hungarian Scientifc Bibliography as well accompanying with publications in non-indexed journals.

 

4. ‘The manuscript uses various subject fields of study in different instances, but it is unclear how the authors conducted specific frequency counts given the potential incompatibility between the subject categories of the Hungarian Scientific Bibliography Database and Scopus. The authors should clarify how they conducted subject field analysis and provide more accurate information.’

Due to the nature of our analysis, we had no other choice but to use the subject field classification defined by the Hungarian Academy of Sciences. As for Scopus, our analysis was limited to a quantitative investigation, that is, how many articles written by the winning applicants found there.

 

5. ‘The paper references five publishers based on citation [34], but considering the rapidly evolving publishing industry, it would be more appropriate to cite more recent studies. For instance, the authors could reference the study by Kim and Atteraya (2023) titled "A decade of changes in OA and non-OA journal publication and production" published in the Journal of Librarianship and Information Science, volume 0, issue 0, with page numbers. The DOI for the article is:

https://doi.org/10.1177/09610006231179718’

Thank you, the paper is closely related to our field of research, we found a way of making a reference to it in the passage about the rapid growth n MDPI publications.

 

6. ‘The last sentence of the conclusion lacks context and does not clearly explain the researcher replacement training program or its relation to the research findings. The authors should provide a more detailed explanation to improve clarity.’

Thank you for your comment. We removed the last sentence from the article as it truly lacks context. Originally, it would have been a piece of advice aimed at university managements in Hungary to plan the creer path of researchers in a more conscious way but as we did not write about its background and about why we would think it is important in the article, we decided to omit the last sentence altogether.

 

7. ‘The conclusion should synthesize the main findings of the research and offer deeper analysis and implications. Currently, it mainly presents the findings without providing a comprehensive synthesis or discussing the broader implications of the study.‘

Thank you for your comment, we modified the Conclusion section so that our hypotheses would be discussed in a more direct way.

Reviewer 3 Report (New Reviewer)

One major drawback of the current study is that it does not properly explain the importance of the study. This factor is neither explained by the abstract nor the introduction sections. Consequently, I still cannot understand the originality of this research. 

In addition, I found many vague statements. Please see the following comments to improve this article further. 

 

Abstract:

please state major conclusions.

 

Line 97: "open access publications"

Once I saw this, I was confused about the connection of this research with OA. I couldn't understand this until I read the hypotheses of this research. Therefore, please explain the connection at the beginning of the Introduction.

 

Lines 154-158:

This contradicts the statements from 57 to 60!! The previous one says there are only two funding opportunities.

 

Lines 180-184:

What are the specific reasons to select awardees from this particular scholarship? 

 

Table 1 - abbreviation 'HASS':

This is likely to confuse with 'HAS', the abbreviation that you have used before. Recommend to change, if possible.

 

Table 3 - average numbers:

Please recheck the average numbers of HASS, STEM, and STEM.

 

Table 3 - the average number of references '115':

Please recheck this number.

 

Lines 310-311:

They should be approximate figures.

 

Line 311:

Shouldn't it be Table 4?

 

Line 333:

Scopus is not a publication house.

 

Line 335:

Table 4 does not illustrate this information.

 

Line 372:

Should be corrected as 'Scopus indexed publications'.

 

Lines 451-452:

This part of the sentence is vague. Please rewrite this.

 

There were places with vague contents. Please edit this article further.

Author Response

Thank you very much for your valuable comments and observations. After a careful consideration, we sought to incorporate the majority of them into ur article in order to give a clearer explanation of its research aims and raise its overall quality.

‘One major drawback of the current study is that it does not properly explain the importance of the study. This factor is neither explained by the abstract nor the introduction sections. Consequently, I still cannot understand the originality of this research.’

Thank you for your comment. The issue is addressed in the Introduction.

 

‘Abstract: please state major conclusions.’

The abstract was rephrased so that the major findings would be given more emphasis.

 

‘Line 97: "open access publications"

Once I saw this, I was confused about the connection of this research with OA. I couldn't understand this until I read the hypotheses of this research. Therefore, please explain the connection at the beginning of the Introduction.’

Thank you. In our modified version, the hypotheses are included in the end of the Introduction so that the relationship with OA would be much clearer.

 

‘Lines 180-184:

What are the specific reasons to select awardees from this particular scholarship?’

All the specific reasons are listed in the detailed description of the Bolyai János Research Scholarship.

 

‘Table 1 - abbreviation 'HASS':

This is likely to confuse with 'HAS', the abbreviation that you have used before. Recommend to change, if possible.’

Thank you for your comment, it also gave us a bit of annoyance as they are easy to be confused but we could not come up with any better idea for making a clearer distinction between these two abbreviations.

 

‘Table 3 - average numbers:

Please recheck the average numbers of HASS, STEM, and STEM.

Table 3 - the average number of references '115':

Please recheck this number.’

Thank you for your comment. The figures have been recalculated, they seem to be right.

 

‘Line 311:

Shouldn't it be Table 4?’

Thank you, it has been corrected.

 

‘Line 333:

Scopus is not a publication house.’

Thank you for your comment, Scopus’ status has been clarified.

 

‘Line 335:

Table 4 does not illustrate this information.’

Thank you, Table 5 displays the right dataset, it has been corrected.

 

‘Line 372:

Should be corrected as 'Scopus indexed publications'.’

Thank you, it has been corrected.

 

‘Lines 451-452:

This part of the sentence is vague. Please rewrite this.’

Thank you for yor comment, the sentence has been rephrased.

Round 2

Reviewer 1 Report (New Reviewer)

Authors made corrections. The article may be accepted for publication.

Author Response

Thank you very much for your valuable comments and observations. We have completed the English language corrections.

Reviewer 2 Report (New Reviewer)

The authors have made significant improvements in this revision and have addressed most of the concerns I raised earlier. However, I suggest that the authors make the following minor correction:

(line 31-35) "Based on this, the descriptive statistics ... scientific excellence rankings."

The aspect mentioned above is presented within the abstract. However, it appears to be absent from the subsequent discussion and conclusion sections of the paper. Since the abstract serves as a concise summary of the paper, I recommend that the authors reiterate this point in a different manner within the main body of the paper.

The general quality of the English is acceptable. However, there is room for improvement in the phrasing and expressions utilized throughout the paper.

Author Response

Thank you very much for your valuable comments and observations. The comment has been corrected. We have completed the English language corrections.

Reviewer 3 Report (New Reviewer)

Please consider the following comment to incorporate the idea. 

This problem was in the previous version of the article too. Lines 92-93 say there are two funding opportunities in Hungary, but line 173 says there are three funding opportunities. Which one is the correct one? Please do not leave doubts.

I believe that you can improve this further. 

Author Response

Thank you very much for your valuable comments and observations. The comment has been corrected. We have completed the English language corrections.

This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.

Round 1

Reviewer 1 Report

In general, the analysis is conducted using appropriate tools, but the methodology and objectives are not always clear. In some cases the analysis tools are questionable, they are presented to the reader partially or they are inadequate. For example, the analysis of the scientific production of Open Access authors is almost inadequate: it focuses on the production with full open access MDPI publisher, totally neglecting both the position of the publisher beyond the OA nature in the current scholarly publishing enviroment, and the analysis of the authors' OA production ( green, gold, hybrid) regardless of publisher and through granular tools. The SJR metric is presented as primarily used to quartile the journal: this is true (and possibile) for all metrics, but the aim of SJR is another. The age of the authors should be replaced by the academic age. It doesn't seem to me that the use of the ratio between references / papers is used adequately, for example to identify different document types, distinguishing full papers from commentary or editorial articles, more generally it doesn't seem to me to bring any significance to the analysis. The conclusions are quite haphazard and seem to me to lack relevance. More generally, perhaps the analysis should be set up using a whole set of normalized metrics by discipline and using bibliometrics together with other extra-publication or derivative indicators (funding, gender, academic age...). The reference to WoS could be eliminated, even if a small comparison between the data present in the two databases (scopus vs. wos) could be useful. The conclusions should analyze the cohort of winners more deeply and, if necessary, create the clearest possible intervention proposals based on the analysis conducted.

Reviewer 2 Report

The study focuses solely on the publishing habits of winners of the Bolyai János Research Scholarship in Hungary, limiting its generalizability. The lack of consideration of other researchers or awards in Hungary or elsewhere makes it challenging to determine if the findings are interesting to the rest of MDPI Publications journal or will widely applicable to other researcher from the rest of the world. Thus, a limited scope reduces the potential impact of the study and restricts its appeal to a wider audience. If it is being published, it will not gather high citation number.

To enhance the generalizability of the findings, the authors should expand their scope to include other academic awards in Hungary or compare the publishing habits of Hungarian academics in awards outside the country. This would provide a broader understanding of academic publishing trends and increase the relevance of the findings to other populations and settings.