Effects of Pandemic-Related School Closures on Pupils’ Performance and Learning in Selected Countries: A Rapid Review

Round 1

Reviewer 1 Report

Dear authors, 

your article deals with a very interesting and necessary topic, and I was very happy to read it. There are however a couple of things that may require some attention:

• You present your analysis as if it were global, and make the case about the number of states that have had to close their schools worldwide (making explicit reference to Brazil or India, for instance - the absence of China is remarkable). However, after having a closer look at your data, they certainly cannot speak for the world, but for a very limited number of Western countries. This means basically that the data you are using cannot sustain the 'global' claim you make in the argument.
• You speak about the effects that closing schools have on children's learning performances and I am afraid that there are many other factors playing a role, which you do not even mention or account for. The comparison (and variation) from 2019 data and 2020 data cannot be reduced to a single factor (schools are open/schools are closed). In fact, even if you could state a significant decrease in learning performances, you cannot say whether this has taken place because of the closure of schools, or, perhaps, due to other reasons. I think you can make your case stronger if you clearly state the countries you have data for. In the EU countries you're studied, welfare states are quite healthy and it may well be that the families of these children are not struggling to survive after losing their jobs or seeing their income severely reduced due to the economic effects of the pandemic - I cannot be so sure about the US. However, I think that worldwide your conclusion is not sustainable given the data you're working with and the monocausal explanation you're elaborating upon.

Author Response

Dear colleague,

thank you very much for the helpful review. Enclosed is our response. We hope that we have implemented your criticism constructively.

Kind regards

Author Response File: Author Response.pdf

Reviewer 2 Report

There are a few editing issues in first paragraph, but otherwise I found the study to be sound and informative, and the need for rapid review of available studies was well explained.

Author Response

Dear colleague,

thank you very much for this positive feedback!

Best wishes

 

Reviewer 3 Report

Thank you for the opportunity to read your article! I had the opportunity to review the presentation "Effects of Pandemic-Related School Closures on Pupils' Performance and Learning: A Rapid Review".

This article provides a rapid review of the effect of pandemic measures on pupils' school performance.

Based on the above, I would like to make a general assessment of the proposal (relevance, structure, discussion, strengths and weaknesses, etc.).

The methodological argument of the article has an adequate methodological basis, although certain shortcomings can be seen in its deployment.

Although it is a quick review, we do not see the range of dates of the searches, it is pointed out that March 10 has been looked at, but if we do not know the range we do not know the valid dates for the research.

Secondly, there is a lack of searches in databases such as wos or scopus.

In short, a more systematic and rigorous review process would be appropriate. Therefore, it will not be less rapid.

It would also be appropriate to present the research questions: if we do not know what we are looking for, how can we properly orient the review?

It would also be useful to know what inclusion or exclusion criteria the authors used to select these articles.

Regarding the presentation of the results, I think it is appropriate. Although it is always necessary to take into account that there are serious doubts in the sample.

In the meta-analysis carried out, it would be appropriate to include the confidence intervals because the samples are very small.

In the references there are links such as:

https://www.nwea.org/research/publication/technical-appendix-for-learning-duringcovid-19-initial-findings-on-students-read506 ing-and-math-achievement-and-growth/

that do not work and should be revised.

This article presents a good theoretical argumentation but in its implementation there are serious biases.

I hope you have the best of luck in your research.

Author Response

Dear colleague,
Thank you very much for the review. Attached is my response letter. 
Best wishes

Author Response File: Author Response.pdf

Reviewer 4 Report

The article information could be found relevant. Though some improvements could be made:

1. The prepared article is relevant to the education system nowadays but its relevance could be proved using statistical data, especially describing how many schools and in which countries were and still are working on the remote way.
2. The authors are not analysing and describing the scientific problem of the article using the literature review method. Their literature review method is already the research results of their chosen literature. We could state that the article's scientific problem analyses are missing. 
3. The authors present what kind of literature review as research is made, but literature review could be improved by analysing the gathered data from articles comparing their data with each other. At the same time, I could not find the position or opinion of authors about the analysed scientific literature.
4. I would suggest preparing the methodology of data analyses made in this article. It is not clear why authors have chosen just two databases that actually are not highly ranked databases. It is not clear how articles were selected and what method was used to analyse chosen articles.
5. The conclusions could be deeper revealed and be formulated specifically after made analyses. It could help determine the aim of the article in the introduction part.

The article contains analyses that are relevant thus it could be improved and could be published after it. 

Author Response

Dear colleague,
Thank you very much for the review. Attached is my response letter. 
Best wishes

Author Response File: Author Response.pdf

Reviewer 5 Report

Thank you for your study. This is a valuable study in promoting an understanding of   Effects of Pandemic-Related School Closures on Pupils.

However, the paper requires better contact with the existing literature, more clarity concerning the goals, what the data can say, and the possible contribution to the literature.   

Here are a few of the main issues:

-I suggest for the authors to include in a literature review section in deeply to describe any existing gaps or contributions.

- to justify more for a need to conduct the current study,

- to improve methodological quality . Was only assessed conditionally, and it means may have possible biases may arise in the presentation of results.

 

Author Response

Dear colleague,
Thank you very much for the review. Attached is my response letter. 
Best wishes

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

The paper does not make any too general claims now.

Reviewer 3 Report

Thank you for your comments and appreciations. Thank you for your contributions, which have improved the quality of the article.

Author Response

Dear colleague,

thank you very much for the helpful review. We hope that we have implemented your criticism constructively.

Kind regards

Reviewer 5 Report

Thank you for your study.  This is a valuable study in promoting an understanding of Effects of Pandemic.

Author Response

Dear colleague,

thank you very much for the helpful review. We hope that we have implemented your criticism constructively.

Kind regards