Investigating the Impact of the Stratified Cognitive Apprenticeship Model on High School Students’ Math Performance

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

Your study is interesting and current. However, I have a few concerns regarding your methodology and data analysis.

First, it is not clear how many teachers taught these classes. You specify that a single teacher taught each CAM class (lines 161-163). You do not specify if a single or multiple teachers taught each CI class and each SCTM class. Did the same teacher who taught CAM classes teach one of the CI or SCTM class? This must be clarified. Additionally, you do not talk about the fidelity of the treatment. Did you observe classes to ensure that the teachers taught their classes according to their designated group's instructional method? All this makes me wonder about the teacher effect. I would like to see examples of lessons taught to each group of students. The fact is that the students will not learn what they were not taught, and it is essential that the students were provided the same learning opportunities (e.g., to solve the same problems, answer the same questions) so that it is clear that only the instructional approach was different.

Second, I am confused by the lower level and upper level classes. I assume this was done for SCTM class only? Or you also had the lower and upper classes for CI and CAM too? The figure at the beginning of page 6 does confuses me more. Did you first divide students into control and treatment groups then implemented the pre-test?

The greatest concern I have is using ANCOVA to analyze data. Yes, it seems logical to use it since you have the pre-test and you want to control the differences. However, there are assumptions that must be satisfied to be able to use ANCOVA and get meaningful results. You do not discuss the assumptions, so how can I be sure that the results you got are valid. For example, you used the pre-test to control for differences, but you did not consider the difference between the lower and upper class, which got instruction at different levels. What if the SCTM worked for lower level class but didn't really work for upper level class? By combining them into single group you made it impossible to examine a possible difference. Also, while you randomly assigned students to each group, the students learned the concepts within intact classes, so your unit of analysis should be a class, not a student. In other words, by using 150 as your sample size instead of 6, you increased the chance to get the significance of your results.

Author Response

please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

 The article presents a very pertinent research focused on a relevant issue: the effects of three teaching models on the mathematical proficiency. The study clearly gives evidence of the superior effectiveness of the SCTM model that combines the cognitive apprenticeship model with the stratified teaching. It presents an adequate research design, and the results are very consistent.

 

I recommend the following alterations:

 

p. 2, l. 74 – explicit what is tacit knowledge

p. 3, l. 95 – what are “CI strategy groups”? The article refers only CI groups and does not mention strategy group

p. 3, ll 124-126 – clarify what is the approach advocated by Manning: CAM or SCTM? From the sequence of the previous sentence that addressed CAM, it seems to refer to CAM, but from its content associated to diverse educational needs, it seems to be SCTM. It is also strange to refer to the approach as adequate for both traditional and innovative models when the traditional model is distinct from CAM model or SCTM model which integrates CAM.

l. 456 – insert a section title because the two paragraphs presented in lines 457 to 485 present a global analysis which is not related exclusively with application analysis (section 3.4)

ll 472-482 – develop more deeply the way the authors consider ZDP theory related with stratified teaching, because it could be used as a counterargument: this theory valorises the scaffolding of teachers and also of peers; therefore, it can justify the importance of the interaction between students with different levels of mathematical competence.

ll 483-490 – include the dimensions of communication and investigating patterns since are dimensions in which the SCTM model revealed be more effective.

ll 491-499 – this section contemplates the critical thinking which was not assessed in the tests applied and not referred before in the article.

ll 501-502 – the sentence is incomplete or must be reformulated. 

The sections 2.2.1 to 2.2.4 must be presented in the previous section of Introduction because they present the theoretical definitions of mathematical dimensions of students’ performance. Sometimes, the authors define the concept with the same term, as are the cases of knowing (gaining knowledge), and communication.

 

The last paragraph of section 2.3 is not related with the tests. Therefore it must be placed in another section.

I suggest that the authors add the scale used to score the tests: 0 to 20? It is important to explicit this in order to understand the mean values. 

The section 2.4 must include the use of ANCOVA.

Correct the way the references are in the text: several cases of several authors do not present “et al.” (for example, Manning); “Collins et al. (1989)” in p. 3, l. 126, must be corrected because the authors must include the reference number and not present the publication date.

Some references are not well formatted.

Author Response

please see the attachment.

Author Response File: Author Response.docx

Reviewer 3 Report

Comments and Suggestions for Authors

·   

 ·      The manuscript is nicely done in general. However, I have a few places where improvements could be made:

 

     On page 3, it is not clear what the authors mean by “knowing and understanding.” I recommend that the authors be clearer about what they mean by “knowing and “understanding” very briefly before they get to explain this in detail in later sections.  

·      I suggest elaborating on Section 1.2. The authors need to explain in the beginning of the section about the hypotheses with one or two sentences at least instead of giving the hypotheses without any explanations of them. Authors could state something like, “we conducted our study with the following hypotheses.”

·      It was unclear to me as a reader how the delayed post-tests were intended to be used in addition to the post-test results until I got to the result section. A brief explanation of this in the beginning of the section would be helpful. Even in the result section, it was unclear why one of used over the other in the analysis. For example, the authors used post-test results in Table 2 and delayed post-test results in Table 3. But is it not clearly stated the reasons for this. A clearer explanation of this would be needed.  

·      The results reported in Sections 3.5, 3.6, and 3.7 are in a level different from the previous sections. It is critical to refer to the specific results in the previous sections to draw the results in these three sections.

Author Response

please see the attachment

Author Response File: Author Response.docx

Reviewer 4 Report

Comments and Suggestions for Authors

The title is not clear. Please re-write the title so that it is clear and understandable.

 

p. 1 Line 26: What is ‘shadow education’ ?

 

p. 1 Line 37: Please remove “Professor” and “from Beijing Normal University”.

 

What is the difference between the CAM and SCTM? Are they the same concepts? Obviously not. But, please clarify these concepts better so that the reader can understand these ideas better.

 

Rationale: The paper does not have a strong rationale. The authors have a stronger case for the significance of the study. 

 

Lit Review: It is relatively weak and needs to be expanded. 

 

Research questions & Hypotheses:  In educational research, we ussually have research questions. If there is a need to have hypotheses we do not necessarily write research questions. In the present paper, there are 10 hypotheses, I do not thing they are necessary. The research questions would be enough. Just the research questions should be written seperately. There is just a single research question, referring to multiple research questions. So, please re-write the research questions. Basically, have 3-4 seperate research questions accordingly.

 

p. 3 Line 126: “Collins et al. (1989) identified six core features of CAM (figure 1)” But, underneath figure 1, we have the following figure caption: “The Six Core Strategies of the Cognitive Apprenticeship Framework (Kurt, 2021)”. so, Collins et al. (1989) or (Kurt, 2021) ???? 

 

 

p.5 line 153: Why did you choose a international baccalaureate (IB) high school? Your rationale? 

 

p. 5 line 156-158: What is the difference between the SCTM group and the CAM group regarding the classroom implementations? I could not understand the difference at all. Please elaborate on this. 

 

Formation of the treatment and control groups should be re-written. It is not clear. How were the groups formed? Randomly? How?

 

 p. 6 Figure 3: There is only a-two week gap between the posttest and the delayed post-test. I feel that two weeks is too short for this purpose. How did you decide the two weekk gap? 

 

P. 6 Line 172: “Mathematics Teaching Experiment Design” The present study is an experimental design. Teaching Experiment Design is something totally different.  

 

p.6 Line 173: “An 11-week mathematics teaching experiment was conducted in this study. B” but on Figure 3, we read that the post-test was conducted 8 weeks later. So, which one is correct?

 

p.6 line 183-185: “This study established a comprehensive framework to examine the multifaceted aspects of student performance within the realm of IB mathematics, guided by the spe cific criteria set forth by the IBO in 2023.” ==> So, it seems that the IB system has a role in this research! This should have been covered in the Intro & Lit. Rev section. 

 

p. 6 line 185-188: “Performance encompasses a holistic evaluation of students’ physical, developmental, and academic achievements, extending to their mathematics proficiency as defined by IBO’s comprehensive criteria, which include comprehension, expertise, communication, exploration, and application.” ==> What is the  point of this sentence? I do not think that this sentence is relevant to this study at all.

 

p.7 line 228-231: “Proficient IB-certified educators devised pre-, post-, and delayed post-tests, and their validity was verified and refined by university experts. A pilot study was conducted to validate the difficulty coefficients and the reliability of the tests. All the test papers met the scoring criteria.” ==> These sentences do not provide the reader with the necessary details about how the tests were constructed. We also do not get to know whether a systematic scientific methad was followed during the test construction. Sample questions are missing.

 

p. 7 Table 1: Are there only 4 items on the tests? The same items on the tests? 

 

p.8 line 256: You need to provide more details about the pilot study?

 

Overall, the research methodology is not clearly explained with all details. I certainly do not see enough evidence of a systematic scientific research process. It is not clear. 

 

There are two treatment groups but there is not enough information about such groups. 

Also, there should an elaborated description of the Control Group Instruction.

 

I personally do not think that this paper can easily be understood by potential readers. there are so many confusing ideas and descriptions. Also, its organization does not represent itself as a systematic scientific research study. 

I feel that the paper has to be re-written. Particularly, the Methods section should be way stronger. 

Author Response

please see the attachment.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

Your additions clarified a lot. However, you did not respond to my argument that the unit of analysis should be a class, 6, instead, a student, 150. You should explain in the paper why you decided to use a student as your unit of analysis.

Also, I did not notice this in the original article, but the tables 8, 10, 12, and 14 have titles "Descriptive Statistics and ANOVA Results for ..."  I believe this is a typo.

Author Response

comments 1 Your additions clarified a lot. However, you did not respond to my argument that the unit of analysis should be a class, 6, instead, a student, 150. You should explain in the paper why you decided to use a student as your unit of analysis.

Response:Thank you for your insightful feedback and for the opportunity to further improve our manuscript. We appreciate your attention to detail and your suggestion regarding the unit of analysis.To make this change clear, we have highlighted the relevant sections in yellow in the revised manuscript. This change ensures that the analysis aligns more appropriately with the study's design and objectives.

Comments 2:Also, I did not notice this in the original article, but the tables 8, 10, 12, and 14 have titles "Descriptive Statistics and ANOVA Results for ..."  I believe this is a typo.

Response 2:We have corrected this error in the revised manuscript. The table titles now accurately reflect the content presented in each table.Thank you for your valuable feedback.

Reviewer 4 Report

Comments and Suggestions for Authors

The authors have addressed my earlier concerns in the revised manuscript. It now is a much better work. 

Author Response

Dear Reviewer,

Thank you for your valuable feedback and for taking the time to review the revised version of our manuscript submitted on 02 July 2024. We are pleased to learn that the revisions have addressed your earlier concerns and that the manuscript is now deemed a much better work.

We greatly appreciate your comments and the opportunity to improve the quality of our research. Your insights have been instrumental in enhancing the clarity, coherence, and overall presentation of the paper.

Should you have any further suggestions or require additional modifications, please feel free to reach out. We are committed to ensuring that our work meets the highest standards of academic rigor.

Thank you once again for your thorough review and constructive feedback.

Best regards,

Wang Ruimei