Science Achievement of Multilingual Pupils: A Study on the Effectiveness of a Read-Aloud Assessment Accommodation

Round 1

Reviewer 1 Report

Please see attached. I think the issues I raise can easily be addressed.

Comments for author File: Comments.pdf

Author Response

We appreciate the reviewers' suggestion to enhance the literature review. In response, we further explored the factors discussed in the manuscript within the existing literature to provide a comprehensive framework for the study's focus on read-aloud accommodation. Regarding the rationale behind the selection of read-aloud accommodation as the focus of our study, we acknowledge the importance of clarifying the decision within the literature review. We provided a more detailed explanation of why a read-aloud accommodation was chosen. In summary, we opted for read-aloud accommodations because they are commonly used by teachers in Flanders, reflecting their acceptance and integration into educational practices. As opposed to some other accommodations, there is little controversy surrounding the use of read-aloud accommodations, making them a suitable focus for our investigation into their effectiveness in supporting multilingual pupils during assessments.

Regarding the second research question, we appreciate the opportunity to clarify the rationale behind the selection of characteristics for our multilevel study on the effectiveness of read-aloud assessment accommodation. We expanded upon the literature review to justify the chosen characteristics in relation to their potential impact.

Reviewer 2 Report

The study on read-aloud accommodation administered to 1,022 5th-grade pupils in 36 Flemish primary schools has relevance to the field of science education of multilingual pupils though the average use of the available audio support to reflect read-aloud accommodation was rather low.

Though the methodology and analyses were detailed, the literature review could be enhanced. Could these factors mentioned in the discussion be reviewed in the literature to explain why the study was focused on a Read-Aloud Assessment Accommodation though there are other accommodation possibilities – ‘In cases where the vocabulary is unknown to the pupil, read- 321 alouds may not be the most suitable accommodation to address MP needs. According to 322 the simple view of reading23, two interrelated components that develop at the same time 323 are necessary: decoding and language comprehension. Accommodating for one – in this 324 case, decoding – may not have the intended benefit of boosting the language compre- 325 hension. In other words, compensation for decoding difficulties may improve reading 326 comprehension but only if language comprehension is already adequate12. Cohen and 327 colleagues24 also suggest that effects may be different when accommodations are explored 328 for older pupils compared to younger pupils, and that more research is needed in terms 329 of cognitive load. Indeed, there are more cognitive resources required in a test with both 330 visual and oral “distractions”.

This is necessary because this constitutes your first research question -  We assessed the hypotheses, first, whether pupils in a condition with accommodation 87 perform be􀄴er than their non-accommodated peers,

As for your second research question, could your literature review explain how and why you have selected the various characteristics for your study to reflect a multilevel study on the effectiveness of a read-aloud assessment accommodation  - which background char- 88 acteristics are related to science achievement.

Was - is used. 183

Author Response

Thank you for your thoughtful and constructive feedback on our manuscript. We appreciate the time and effort you have invested in reviewing our work. We have carefully considered all your suggestions and critiques, and we believe they will significantly enhance the quality and clarity of our paper. We have made the necessary revisions as per your recommendations, and we are confident that the revised version will meet the standards of the journal. We are grateful for the opportunity to improve our work based on your invaluable insights.

1. We understand your perspective and would like to clarify that we opted for read-aloud accommodations because they are widely used by teachers in Flanders and are well-integrated into educational practices. Unlike some other accommodations, read-aloud practices have faced minimal controversy, making them a suitable focal point for our investigation into their effectiveness in supporting multilingual pupils during assessments.

While researching the use of learners' L1s as accommodations is certainly valuable, our study aimed to address the specific context of Flanders, where read-aloud accommodations are prevalent. Nonetheless, we acknowledge the importance of exploring various language-related accommodations and their potential benefits, and we appreciate your suggestion for future research directions.

2+3. Thank you for your comment regarding the obviousness of the result showing that L1 speakers of the language of schooling outperform multilingual pupils. We acknowledge your observation and agree that this outcome may not be surprising to readers familiar with language acquisition dynamics. In response to your feedback, we have deleted the reference to this result from the abstract to reduce the focus on it and highlight other significant findings of our study.

3. Regarding the term "multilevel" in our title, we've considered your feedback and have removed it to ensure clarity and appropriateness.

4. We've taken note of your suggestion to provide a clearer definition of "multilingual" and to elaborate on the linguistic background of the learners in our study. In response, we have defined multilingualism in the introduction and included additional information about the pupils' linguistic backgrounds. Thank you for bringing this to our attention.

5. Indeed, both tests are Dutch only. We have rewritten this part, as it was confusing. Thank you for pointing it out.

6. Agreed, we appreciate your insight. We have acknowledged this disadvantage by adding the following point: "However, one disadvantage of recorded accommodations is that learners do not have the visual cue of seeing the person's lips/mouth move, which can be beneficial for understanding, especially for multilingual learners."

7. Yes, the Trends in International Mathematics and Science Study (TIMSS) is a curriculum-based assessment. It evaluates students' knowledge and skills in mathematics and science based on the curricula taught in participating countries. The assessments are designed to measure how well students have mastered the content and concepts outlined in their respective national curricula. TIMSS aims to provide insights into the effectiveness of educational systems by comparing student performance across different countries. See for example: TIMSS | IEA.nl

8. We agree with your suggestion to provide clearer implications for further studies in our conclusions section. We have rewritten the conclusion to address your comments. We hope this revision adequately addresses your concerns. 

Round 2

Reviewer 2 Report

Accept without need to revise.