Evaluation of the Collaborative and Proactive Solutions Model in an Alternative Educational Setting

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

This is an interesting and important research topic. There is potential to have better results if the authors could improve the research design and exert better manipulation of the intervention and extraneous factors that contributed to the change of the dependent variables.

I want to list several major concerns regarding the study

1. As the author already pointed out, the limitations of the research design are that only two student participants complete the study which nullifies the multiple baseline design. If you can recruit more participants or change a research design, it would be better.

2. The fact that neither of the two student participants has the proposed solution implemented for step c in the Collaborative and Proactive Solution model may put the study in question as not the full intervention was implemented as planned. This study can serve as a pilot study and if the researchers could continue with the study and have the full intervention implemented, it's probably to get plausible results.

3. The author did not provide a clear rationale for using two dependent variables off-task behavior measured during the observations and direct behavior rating of the students' off-task behaviors. The two dependent variables are highly correlated. It's better to choose one of the two variables as the main dependent variable.

Comments on the Quality of English Language

There are some minor language and grammar errors that need to be fixed. Some of the errors make it difficult for the readers to understand the main results. For example, in the third last line in the result section 3.3 "Sam "would not learn" ten transitions".

Author Response

Dear Editor and Reviewers,

Thank you for your constructive review, which greatly contributed to the improvement of the manuscript. Below, in the attached file, we present our revisions related to each reveiwer’s commentary.

Best regards, 

Authors

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

Abstract Suggestions for Improvement:

Clarity and Conciseness: While the abstract provides valuable information, it could be more concise. Consider condensing certain parts to maintain brevity.

Detailed Results: Provide a bit more detail on the results of the study, such as the specific findings of the systematic direct observations and direct behavior ratings. This would give readers a better understanding of the study's outcomes.

Context and Significance: A brief statement about the broader context and significance of the study could be beneficial. Why is it important to evaluate the CPS intervention in this alternative educational setting? What implications might the findings have for educational practices or policies?

Quantitative Data: Include quantitative data, such as percentages or effect sizes, to support the findings. This would provide readers with a more concrete understanding of the results.

Introduction Suggestions for Improvement:

Research Objectives: The research objectives could be more explicitly stated. For example, a brief statement on how the study contributes to addressing the challenges mentioned in the introduction and why evaluating CPS in an alternative educational setting is essential could provide a stronger foundation for the research questions.

Terminology Clarification: While the introduction provides various terms such as "social validity," "acceptability," and "usability," it would be helpful to offer brief definitions or explanations for these terms to ensure reader comprehension.

Flow Improvement: The last section of the introduction transitions to discussing specific aspects of the study (e.g., focusing on social validity) but could benefit from a smoother transition.

In conclusion, the introduction effectively sets the stage for the study, presenting the context, research objectives, and research questions. To enhance it further, consider explicitly stating the research objectives, maintaining consistent citation style, clarifying data sources, explaining key terms, addressing ethical considerations, and improving the flow between sections.

The Materials and Methods section of this scientific article provides a detailed description of the research methodology employed to evaluate the effects of the Collaborative Problem Solving (CPS) intervention on children's off-task behavior. Overall, the section is well-structured and provides a comprehensive understanding of how the study was conducted. However, there are a few points to consider for improvement:

 

Clarity and Consistency: The section could benefit from improved clarity and consistency. For example, it introduces "Bill" and "Sam" as participants in the study without specifying that these are pseudonyms. It might be clearer to establish the use of pseudonyms right at the beginning of the participants' description.

 

Justification of the Methodology: While the article explains why a single-subject experimental design was chosen, it would be helpful to provide a more comprehensive justification for the choice of this methodology. What are the advantages and disadvantages of this approach in this context?

Ethical Considerations: The article mentions ethical approval and informed consent, which is essential. However, it would be helpful to elaborate on how the researchers ensured the ethical treatment of participants, especially children. Did they use child-friendly language in obtaining consent, and how did they ensure the well-being and privacy of the participants?

Participant Characteristics: The descriptions of the two participants, Bill and Sam, are informative. Still, the section might be improved by including more background information about them, such as their academic history and any prior interventions or treatments they received, which could affect the results of the study.

Data Collection: The description of data collection methods is detailed and thorough, but there could be additional information about the instruments used and their validity and reliability. This would enhance the reader's understanding of the data collection process.

Implementation Fidelity: The article describes the implementation fidelity of the CPS model and how it was measured. However, there's no mention of the results or the implications of the fidelity scores. Were there any deviations from the expected fidelity, and how did they impact the study?

Acceptability and Usability: The section about acceptability and usability provides important insights into how participants perceived the intervention. However, the results of these assessments are not included. Including the findings would make this section more informative.

Data Analysis: The section discusses the data analysis techniques, specifically the Tau-U statistic and visual analysis. While it mentions what Tau-U scores indicate, it could be useful to explain why these methods were chosen and how they are interpreted in the context of this study.

Results Suggestions for Improvement:

Clarity and Organization: The text is relatively clear and well-organized. It starts by providing an overview of what is being discussed and then proceeds to delve into different sections. However, it could benefit from subheadings to make it even more organized and easier to navigate.

Precision and Detail: The text provides a detailed account of the interventions and outcomes for Bill and Sam. It's important for scientific articles to be precise and thorough in their descriptions, and this text does that effectively.

Statistical Analysis: The text includes some statistical analysis, which is an essential aspect of scientific research. However, it would be beneficial to include more information about the statistical tests used and why they were chosen. This would provide transparency and help readers assess the validity of the findings.

Extraneous Events: The text mentions "extraneous events," such as Bill's extended placement in an alternative educational facility and Sam's new mobile phone, which affected the interventions. It's important to discuss these in more detail to provide context for readers and explain how they might have influenced the outcomes.

Qualitative Data: The text includes qualitative data from interviews, which can provide valuable insights. It's important to include quotes or examples from these interviews to illustrate the students' and teachers' perspectives.

Dicussion Suggestions for Improvement:

Clarity and Organization: The discussion is well-structured and logically organized. It starts by summarizing the study's motivation and the research problem, followed by a clear discussion of the results. This is a good practice in scientific writing.

Integration of Literature: The discussion effectively integrates relevant previous research (citations [3, 4, 7-8, 14, 17, 19, 23, 24, 27-30, 31, 32, 36, 37, 38, 39, 40, 42]). However, for better clarity, it might be beneficial to discuss how the current study relates to and extends upon this prior work more explicitly.

Statistical Findings: The discussion provides a good overview of the statistical findings regarding the effects of the CPS intervention on students' off-task behavior. This is a vital aspect of a scientific discussion section.

Qualitative Data: The inclusion of qualitative data from interviews with students and the teacher assistant is valuable, providing insights into the social validity and usability of the CPS intervention. However, it would be beneficial to provide specific quotes or examples from these interviews to illustrate the points being made.

Interpretation of Null Results: The discussion does an excellent job interpreting the null results, suggesting that the number of problem-solving conversations and the fidelity of implementation might have influenced these outcomes. This is crucial in scientific discussions as it helps readers understand why the expected effects were not observed.

Consideration of Extraneous Events: The discussion acknowledges the potential impact of extraneous events on student behavior, providing a thoughtful analysis of how these events might have affected the outcomes. This demonstrates a thorough understanding of the research context.

Generalizability and Limitations: The limitations section is clearly presented and provides a cautious interpretation of the results. Mentioning the small sample size and deviations from the intended research design is essential for transparency.

Recommendations and Implications: The discussion might benefit from a section that explicitly states the implications of these results and provides recommendations for future research or practical applications. This would help readers understand the broader significance of the study.

 

Terminology: Ensure that any specific acronyms or terms are clearly defined for readers who might not be familiar with the field or the study's context.

Author Response

Dear Editor and Reviewers,

Thank you for your constructive review, which greatly contributed to the improvement of the manuscript. Below, in the attached file, we present our revisions related to each reveiwer’s commentary.

Authors

Author Response File: Author Response.pdf

Reviewer 3 Report

Comments and Suggestions for Authors

Thank you for the opportunity to read the manuscript. I really enjoyed reading it and I believe it represents a timely and valuable contribution!  

Introduction

Please consider to add a research question/ hypothesis  that can guide the reader through the text already in the end of introduction. There are a couple of statements which needs to be supported by ref ( intro sentance 2). 

Theory 

The paper  discuss the main concepts properly.  However,  the concept children with behavioural challenges should be  clarified.  I may  critically remark that term challenges  is something everyone has. Indeed, the term  you use may encompass the normal challenges of everyday existence. It is unclear for reader  what kind of behavioral challenges are you talking about in this paper? Are those students  diagnosed as having behavioral disorders / disability?  or are talking bout students with disciplinary problems? This information comes only in section  2.1. Participants and Setting. It would be helpful to discuss the terms used already in the sencion theory. 

Author Response

Dear Editor and Reviewers,

Thank you for your constructive review, which greatly contributed to the improvement of the manuscript. Below, in the attached file, we present our revisions related to each reveiwer’s commentary.

Best regards, 

Authors

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

The two main concerns of the study is on the research design and the results. As the authors point out with only two participants, the multiple baseline design could not support the functional relationship between the intervention and results. Another shortage is none of the participants really complete the implementation part of the CPR model. 

Comments on the Quality of English Language

The English language was fine. No big problem.

Author Response

Dear reviewer,

Thank you for your comments on the manuscripts discussion and conclusions. We have now revised these parts. As we could not ascertain a functional relationship between the intervention and the results, we decided to delete to paragraphs discussing the effects of the results of the intervention. Instead, in the first discussion paragraph, we added that we cannot draw any conclusions of the effects of the intervention due to the design, but the results can be used when planning future CPS interventions in school settings. This study is important since it is the first of its kind in a school setting and we therefore believe the results are of interest to the audience of the Education Sciences journal and can be published as a brief report. 

Best regards,

The authors