Tracing Progress in Children’s Executive Functioning and Language Abilities Related to Reading Comprehension via ExeFun-READ Intervention

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

Please refer to the tracked comments found in the manuscript.  

Comments for author File: Comments.pdf

Comments on the Quality of English Language

The quality of the written language is generally satisfactory though there are some points that need clarification.  

Please refer to the tracked  comments for further action.  

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

Dear authors, the title is illustrative and compelling, and the abstract is succinct and comprehensive to reflect the entirety of the paper. I have no concern with the originality and quality of the paper but for a very faint issue of the gender consideration as to the respondents. Did that prove of any research relevance? 

The conclusions are concise and short, stemming from the discussions part and there is a pertinent approach to the limitations. 

Author Response

Dear reviewer, thanks a lot for positive assessment of the paper.

In order to increase methodological clarity of the paper, the introduction, methodological aspects and discussion were elaborated in the article.

In terms of your question: ....issue of the gender consideration as to the respondents. Did that prove of any research relevance?  

Answer: 

The dependent variables in the experimental study were (1) executive functioning processes (inhibitory behaviour, cognitive flexibility/switching, processing speed, self-regulation, and attentional control), and (2) language skills, considered as subcomponents of reading comprehension (semantic knowledge, syntactic knowledge, verbal fluency/inferencing, verbal analogies). School performance was the controlled variable.

Unfortunatelly, in this paper, the gender was not controlled as variable, the deeper analyses of the data are planed, gender as a variable will be included.   

Thanks once more.

Reviewer 3 Report

Comments and Suggestions for Authors

 Current study is about the relation between reading comprehension and executive functioning. I appreciate that the authors study this relation, and I value the idea of empirically studying intervention studies in this field. However, I have some concerns, outlined below.

 

Abstract

-       Interesting research topic, highly relevant to find out more about the relations between EF and RC. 

-       First sentence of the abstract is confusing. Is this research about low-achieving children? Why? And what is meant by ‘working with information in a text’? Could you rephrase the first sentence, maybe with less jargon?

-       So two outcome measures, namely RC and EF? Or also language skills?

 

Introduction

-       The theoretical framework would gain from a more comprehensive (state of the art) literature review on executive functioning and reading comprehension and their relation. How were concepts defined and conceptualized, what was already established, more specifically on the (inter)relation, what do we know about the impact of intervention on these outcome measures, what remains understudied?

-       Please add a theoretical insight in the population of interest in current study. It is not clear why low-achieving students were studied, neither why this age group is of interest for current study aims. 

-       Please, consider one term for your studied group. Is it low-achieving students, or low-performing pupils? 

-       Could you also explicitly conceptualize/define what you consider low-achieving children? Although a footnote was added, I would expect a definition in text too, since it appears of great importance for your current design. It is, for example, not clear to me why you want to study this intervention specifically in children with low performances in mathematics, please make sure that the theoretical part of your study explains your aim/interest in this population?

-       P2 r70. I would not expect a design in the second paragraph of theoretical framework. Consider to move this towards last paragraph of introduction, or the method section.

-       P2.r.73. nothing was explained yet about what ExeFun actually does. Please, first introduce what interventions are studied in previous literature, what components you are interested in (and why), and how this might interrelate with RC and/or EF.

-       P2.r.76. hypotheses should flow from introduction. It is not clear were these hypo’s come from (generated by theoretical considerations? Or by previously reported studies?). Please embed.

-       P2.r77. Introduce what is meant by inhib, flexibility,self-regulation, and attentional control.Theoretical embedding would help judge current operationalizations, and contributes to comparison with other literature and empirical studies.

-       P3r87. Same goes for the skills/knowledge mentioned related to semantic, syntactic, fluency, and analogies. How would you define them; what do we know from literature about how it impact RC and how it could be trained? 

-       P2.79. authors compare to active and passive conditions, but in this phase of the paper it was not yet explained what these conditions comprise. Not easy to understand for the reader, therefore.

-       It seems to me that the intervention tries to capture a whole (holistic?) bucket of skills and knowledge. Please specify the characteristics of the intervention. Try to explain why you expect this intervention to cause growth/development and why the two other interventions (controls) do not? What is the holy grail of this intervention? 

-       P3.r90. At first, I understood that the authors were interested in two outcome measures (namely RC and EF) of the intervention. Now, it appears that also the Slovak Language skills were aimed for. This should not by introduced this late in the introduction section. Please move from hypo’s towards theoretical framework, and explain what was already known in literature about this hypothesized relation. In addition: what was the relation of interest? I can imagine that the authors are interested in how EF is related to the individual variation in Rc, and indeed it would than be relevant to control for decoding and language skills (e.g., Nouwens et al 2020). Was that indeed the purpose of the other measures? Please clarify and make clear how this is used in the chosen design and analyses.

-       To my opinion, the introduction section is not complete yet. The scientific and societal problem is not completely addressed and it is not fully clear what key concepts are studied/related and why it deserves this new research. Introduction section would gain from more focus on the key issues and theoretical frameworks in the field, and the literature should lead towards the (tested relations in) research questions and hypotheses.

 

Method

participants

-       P3.r100-.. It is not clear to me why students were ‘equalised’ in terms of reading speed, error rate, etc. Please explain why decoding was an issue in the sampling of this study about reading comprehension skills. Consider explaining the theory of the simple view of reading (Hoover & Gough 1990) in the introduction section?

-       It is not clear why first trio’s were made? And how it was thereafter accounted for by gender, math, and language? Do authors mean they matched samples based on certain criteria?

-       Please, give participant information per sub sample. It would help to know, per condition, what the characteristics (age etc) and mean levels and scores were. Please, also add statistical tests to show the comparability of the groups on the relevant measures.

Design/procedure

-       What is the conceptual model (visualize?) of current study? How would you organize the great amount of measured variables and how are they (conceptually) related to each other (direct? Indirect?).

-       Given the complex method to assign children to the three conditions, I do not fully understand the procedure. Were all three conditions present in one classroom? Was the intervention administered in classroom setting? What did the children in group 3 do while the others were working with the extra lessons and materials?

-       How/where/by whom were the pre- ant post EF and language tasks measured?

-       Please, add information about consent procedures, information letters/transparency to participants, data management, ethical approval etc. 

Materials

-       P4r133 How is the score calculated? And is it one latent score for EF in the end, or 4 separate scores? Same for language measures: how were scores calculated?

-       P4.r137. Please, also add some information on what the tasks were, how long it took/how many items, reliability, and validity? 

-       P4.r167. I do not understand this sentence. How was visual perception stimulated for example?

-       P4.r172. How were the questions coded? What question was targeted at planning and what question was targeted at inhibitory, for example? It is not clear to me how the intervention materials work.

-       P5r179. Why these 5 components? I do not understand why self-regulation was a separate module. I thought the authors aimed for many more EF-skills? I do also not understand why word was a component? What do the authors mean by ‘Word’? is it about vocabulary training, or word decoding skills, or other?

-       P5r184. This information is new for the reader. I think it should be clear earlier in the paper. What is meant by the ability to move to active teachers (the third goal) and why is this a goal in current intervention?

-       P5r188. This is new information. Why is peer tutoring of relevance for current intervention? If of relevance, it should be explained in theoretical framework.

-       P5.r188. think aloud method comes out of the blue. Please embed choices in theory.

-       Why is 2-3 times per week recommended?

-       I would advise to choose focus in current study. Too many variables, concepts, constructs, and aims were mentioned. It is not clear for me as a reader what elements were studied and how and why. 

-       Restructure the method section (APA style). Information is not clustered now, which makes it not easy to follow.

-       Table 2. Where do the recommendations stem from? Please legitimize. 

-       Methodological quality and integrity is not clearly reported.

-       Please add a section for analysis plan. Include the planned analyses and their assumptions, power-analyses, handling missing data, significance levels, planned post hoc analyses, etc.

-       Method section is not complete and not easy to read. Not ready for replicatibility purposes for example. Variables were not justified in terms of links with theory/RQs.

 

Results

-       Please start the results section with descriptives of the measured variables and groups.

-       Please add correlation tables and report on relations.

-       Introduce the analyses, by relating them to the RQ’s.

-       I would not recommend to take all separate outcome measures. Statistics are not easy to interpret in the light of the Rqs.

-       P7.r219. it was interpreted that progress was found, but reader cannot see since descriptives are missing. Same for interpretations in r.223.

-       How were conditions separated? Post hoc analyses? Planned contrasts?

-       Consider visualization in graphic/diagram to show pretest and posttest scores and difference levels of all groups.

-       Please report how table 3 and 4 results were interpreted (at least some remarkable results). For example, does Stroop interference test scores mean that all groups 

-       developed their skills? Was this as hypothesis and why?

-       So many comparisons were made; I lost track. Guide the reader through it (or even better: try latent variables or focus on less outcome measures). Please, relate results to RQ’s and help reader to find how these analyses answer your main RQs.

-       P9.r.267. explain why a different approach was chosen here.

 

Discussion section

-       I would not expect new theoretical information in the first paragraph of the discussion section (APA guidelines). This might already be relevant in previous parts of the paper? Pease, summarize aims and findings here. Relate to literature.

-       Authors repeatedly write about reciprocal effects of EF and RC. It seems a main topic. Please help reader to find this goal in the results section, by explicitly relate to hypo’s.

-       P11.r321. provide insight in what aspect of the intervention might have contributed to this result. Generalize the findings.

-       Please, discuss how you interpret the increase in test scores in the control groups. What might have happened and how is this related to the interpretability of your results and conclusions?

-       Page 11.r.358. so many aspects were aimed in the intervention. Be transparent about what interpretations were speculation of the authors and what part could be concluded from analyses.

-       P12 r401. What might this mean (increase in active control group)? Can you conclude and discuss (explain) something from these (unexpected) results?

-       P12.r.409. what was mentioned above? 

-       I think a limitation of current study is also about the multitude of things. Not all components of the intervention were explained nor embedded in theory. It is not clear what components of the intervention method were expected to make (what) improvements. Analyses do not fully answer this question afterwards. Be transparent about this limitation.

-       P12.r416. I do not understand the role of parents in this intervention? This is new information.

-       P12.r408. have you considered multilevel analyses to study the class/teacher effects?

-       To what extent do you think that the randomization of the three conditions over classes/schools would compensate for implementation differences?

-       Please add implications (societal and scientific) section to discussion. Be clear about what can be learned from current study (and what not). Generalizability.

 

-       I think the conclusions do not yet flow from the results. And I think it is not the complete picture (improvements were not allocated to the intervention group). Please reconsider and phrase.

Author Response

Please, see the attachment.

Author Response File: Author Response.pdf