A Bibliometric Study of Issues in Educational Policy

Round 1

Reviewer 1 Report (New Reviewer)

Comments and Suggestions for Authors

The current paper conducts a bibliometric study on the issues published within educational policy research between 2000-2023. While the topic and results are important and interesting, I have a few significant concerns that prevent me from recommending publication at this time.

Major Points:

11. The introduction/literature review is not focused on the analyses and conclusions. For example, the results do not address “why do we see so many changes in educational systems and schools?” which the opening sentence of the manuscript and many paragraphs are dedicated to the importance of educational policy. The analyses and results, however, uncover the specific research topics, scholars/authors, and peer-reviewed articles that have the most exposure and impact within educational policy scholarship. Rewriting the literature review so that it reflects the importance of knowing and understanding the “bibliometric study” and history of academic scholarship within educational policy is needed to match the research methods and results.

22.  There are several paragraphs that lack adequate citations, including lines 29-38; lines 90-103; lines 444-448

33.  The written description of the Inclusion/Exclusion Criteria is difficult to follow. A write-up that follows the same flow as Figure 1 would be more effective (e.g., starting with 931 documents, then explaining how it got to 837, then 363).

44.  Were there previous bibliometric studies or meta-analyses within educational policy research? If so, what where those findings and how does the current study add/revise the current working knowledge?

Minor Points:

11.  Figure 5 shows an interesting pattern; it seems that 2015-2017 experienced the most diversity in terms of topics covered within the research. A potential explanation for why there were so many different topics covered during this short time span and why certain topics continued while others stopped would be useful.  

22.  Some of the conclusions from this study reflect the publication and professional development trajectory of scholars in educational policy. Explaining the findings within this context could be useful. For example, other topics within educational scholarship may experience a faster (or slower) publication and citation record compared to what was uncovered in this paper. Knowing these comparisons would be helpful when determining the impact and influence of educational policy researchers across the broad field of educational science. This could also be helpful for policy-makers, when they are trying to decide which scholarship to prioritize when amending and forming new policies.

Comments on the Quality of English Language

There are several awkward phrasings and paragraphs throughout the paper, and particularly within the literature review. For example, phrases such as "findings with reality" and "data analysis process were thoroughly elucidated" (lines 237-242) reflect the need for moderate/substantial editing.   

Author Response

Dear Editor,

 

Please find our response to Reviewer 1. Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report (New Reviewer)

Comments and Suggestions for Authors

The article is well-presented; however, there are some areas that need improvement, namely:

1.   Ensure that the abstract follows a cohesive and logical structure, starting from the objectives, methods, results, conclusions, and implications. In the abstract, please review the main objectives of the research or study conducted. Also, pay attention to the implications of the research findings.

2.   Review the discussion section. What is the meaning of your research findings? Are there any findings that do not match the expectations?

3.   Separate the conclusion from the discussion.

Author Response

Dear Editor,

Please find our response to Reviewer 2. Please see the attachment. 

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report (New Reviewer)

Comments and Suggestions for Authors

I appreciate the careful and thorough revisions provided by the authors. While the methodology and results are more clearly articulated, significant gaps between the literature review/states purpose and conclusions still remain. 

From my perspective, their introduction and conclusion needs to be firmly rooted within the context of 'understanding the research landscape' and uncovering the main contributors of this research. All of the previously written paragraphs about the importance of policy or the historicaly lineage of educational policies is too broad for this specific topic. For example, the authors included a section of the importance of data-driven work in establishing policies, and point out that research is often focused on policies that are not effective. These points are more relevant than establishing the fact that educational policies are important and change often. 

The conclusions go on similar tangents. Much of the original paragraphs on page 12 (ICT, UNESCO) are not relevant enough to the specific results of the paper: identifying specific authors and topics that are highly influential within the educational policy research publication space. 

This disconnect continues onto the new paragraphs written in 4.1 Conclusions. Specifially, the paragraphs that begin with "Environmental issues.," "Special education", and "Science education" will be more useful if they were directly connected to specific articles uncoverd by the anlayses. This is done effectively in the previous paragraph ("Interest in... like Ziedler publishing." 

Finally, similar to social network analyses, these results have immediate implications for the impact of specific individuals within the population of educational policy researchers. These findings will be particularly useful for up-and-coming junior researchers and within the context of professional development within the field (as well as diversity of thought among journal editors). Spending some time to discuss the influence of the authors (vs the policy topic they covered) could be very useful for future resaerchers and provide necessary context for policy-makers and grant-providing-organizations who are interested in data-driven ways to support their work.  

Comments on the Quality of English Language

There are still some awkward phrasings and many unnecessarily repetitive statements. While they are minor issues, the paper requires a full read-through from start to finish in order to catch them. 

Author Response

Please see the attached report

Author Response File: Author Response.pdf

This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

 

Thank you for submitting your manuscript. After careful consideration, I feel that it has merit but does not fully meet publication criteria as it currently stands. However, while the topic seems important, therefore, I invite you to submit a revised version of the manuscript that addresses the followings points:
 

1. What is missing in Abstract is a description why this is a relevant topic, and what the theory behind the research questions is. At the moment the paper's aim, the method applied, and the main results remain unclear. In my opinion, keywords need to be revised. At the moment they do not correspond to the content of the article.

2.At the introduction authors should indicate: the problem addressed and why it is important, and should be indicated what is the contribution of this study and what is its innovation for the research area. What is the knowledge gap that this study aims to fulfill?

3. The authors may need to consider how the literature review can be made more consistent with the manuscript title and research questions.
4. The authors address one three research questions, but how the research questions are grounded on the theoretical background remains unclear.
5. The method section could give a better overview of the research design. The instruments' purpose and appropriateness to answer the research questions are questionable. The chosen instruments need to be better justified and explained.

6. The result section, in general, is presented based on research questions. As the data analysis process remains unclear, how such results were achieved is not clear. Thus, the reliability of the results is questionable and raises questions. How were the articles analyzed? How did researchers decide which articles are relevant and which are not? It would be interesting to see the list of analyzed articles. The list is missing. What were qualitative data analysis methods? Examples are missing. How the most global cited document content was analyzed? The analysis process needs to be described appropriately.

7. It is necessary to better explain Figure 4. Figures and tables should be formatted according to the requirements of the journal. At the moment Figure 6. is not readable.

8. In Discussion part authors suggested to “increased publications per author in educational policy” (326). What do you mean by that? How to implement it? Can publication production contribute to better education policy?

11. I would like to see some discussion of the validity and reliability of the measures. The use and selection of instruments and data analysis methods need to be discussed more throughout. The relevance of the findings and the connection between results and conclusions need to be better justified.
I think you can follow the suggestions I have given in this review and make a new version of your interesting paper.

Comments for author File: Comments.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

I feel you have over-focused on process here, rather than examining the context of where your work is located.  What is your theoretical stance?  Why might such work be needed?  What are the objectives of the study?  What might be the most effective methodological process to meet these objectives.  Unfortunately I think this needs to be rejected and rethought.  I would recommend you not focus on methodological tools, but take some time to design your research question with care within a broad overview of the field, using a clear theoretical stance. 

Comments on the Quality of English Language

The expression requires extensive editing and is clumsy in many parts.  You make wide assertions in a dangerously generic way, something we have to be very weary of in  scientific publications.