A Content Analysis of the Algebra Strand of Six Commercially Available U.S. High School Textbook Series

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

Review of Educ. Sci. 2024, 14, x FOR PEER REVIEW

TITLE: A content analysis of the algebra strand of six commercially-available U.S. high-school textbook series

 

I must admit to being initially disappointed with my decision to review this manuscript when I discovered that the six textbook series being analyzed had been published as far back as the years 2007-2009. I had expected and hoped that a paper being submitted in 2024 would be dealing with much more recent textbooks. Had the abstract included this information, I would likely have turned down the request to do this review. However, I would then have missed out on reading what turned out to be a very informative manuscript describing a study carried out with rigor and reported with clarity. While I am still wondering whether or not more recent textbooks reflect on a whole the results of the analysis presented in this manuscript, I do not hesitate to commend the authors for their systematic work in conducting this study. However, this is not to say that a suggested revision (to be outlined shortly) is one that I believe would add even more heft to the manuscript. Nevertheless, the current shape of the manuscript is very, very good.

 

Let me get a more minor issue out of the way first. The copy that I received of the manuscript has a problem with the presentation of Figures 1 and 3. In Figure 1, all of the arrows are missing. In Figure 3, the separators between the two main columns are absent, as well as between the successive three blocks of the right-hand side.  

 

My main suggestion concerns Section 2.2 on Analytic Frameworks, more particularly the subsection on cognitive behavior. This subsection is far too brief, in my opinion, and leads to several questions on the part of the reader. The categories of cognitive processes, overviewed in Figure 4, were stated to be based on the TIMSS Advanced 2008 Assessment Framework. Desirous to know more about the definitions and cognitive processes listed in Figure 4, I scouted out the TIMSS reference. This reference did yield some needed detail, but it was still not clear what sorts of textbook items would be categorized specifically as examples of the given cognitive processes by the authors of the manuscript. For example, was the activity of equation solving categorized under the process, “Solve Routine Problems,” within the Applying category or was it rather considered under the process, “Compute,” within the Knowing category? There is somewhat of a blurring between the classic categories of “knowing that” and “knowing how” with “compute” being included under Knowing and “solving” under Applying. The fact that the Applying category had such a high incidence (as shown in Figure 6) adds another reason for supplying more detail as to what kinds of items were considered to be included under Applying (as well as for the other two domains of Knowing and Reasoning). Furthermore, the process that is named “Recount” was not included in the TIMSS Advanced 2008 Assessment Framework. All of this leads to the suggestion that each of the cognitive processes that were listed in Figure 4, and which were used in the subsequent coding of the textbook items, be accompanied by at least one example of a textbook item that illustrates that particular cognitive process. While such information could be included in the given subsection, another possibility would be to add an Appendix, so as to not break up the rather tight flow and briskness of the manuscript’s narrative.

 

My last comment concerns Section 4 on Feedback from Textbook Series Authors. I thoroughly enjoyed reading this rather unique piece of the analysis. The disagreements that the textbook authors expressed with the way in which the codes, and the methodology in general, were applied by the researchers were presented in an honest and kind of self-deprecating manner. The overall effect was a trickling back to the entire manuscript a sense of its truth value and the care with which the authors presented their data and its analysis. The inclusion of this section very much enriched the reporting of this study on textbook analysis and could serve as a model for other researchers engaging in similar work. 

 

The authors conclude by reiterating “the well-established axiom that students learn what they have an opportunity to learn.” Their textbook analysis reflects the two differing perspectives on algebra that are also encountered in research on the development of algebraic thinking in students – that of a symbolic algebra based on number and generalized arithmetic versus that of a function-based view of algebra. Nevertheless, if their selection of texts mirrors the general state of textbook use in high schools across the U.S. – that is, a majority of textbooks of the subject-specific type (i.e., a 4:2 ratio in comparison with integrated textbooks) and where there is a stronger focus on the symbolic algebra perspective than on the function-based view – the question arises as to whether the preponderance of a function-based focus in the development of algebraic thinking with elementary and middle school students indicates a movement in the wrong direction in that it does not really prepare students for what is to come. But such a question is of a more personal nature and, as the authors rightfully state, their study “leaves open fundamental questions about what understanding and skill in algebra is most important for students to acquire from their school mathematics experience.”

 

Author Response

 

Response to Comments from Reviewer 1

Thank you very much for taking the time to review this manuscript. Please find detailed responses to your comments below. The corresponding revisions and corrections are in the revised manuscript, in which track changes was utilized.

Reviewer 1 Comment 1: The copy that I received of the manuscript has a problem with the presentation of Figures 1 and 3. In Figure 1, all of the arrows are missing. In Figure 3, the separators between the two main columns are absent, as well as between the successive three blocks of the right-hand side.

Author Response to Comment 1: Thank you for pointing this out. Figures 1 and 3 appear correct in the version of the manuscript that we are instructed to use for revisions, per the e-mail on 7-16-24 from Mr. Clement Liu, Assistant Editor. If there is a correction to be made, we require more information and assistance from Mr. Liu.

Reviewer 1 Comment 2: My main suggestion concerns Section 2.2 on Analytic Frameworks, more particularly the subsection on cognitive behavior. This subsection is far too brief, in my opinion, and leads to several questions on the part of the reader. The categories of cognitive processes, overviewed in Figure 4, were stated to be based on the TIMSS Advanced 2008 Assessment Framework. Desirous to know more about the definitions and cognitive processes listed in Figure 4, I scouted out the TIMSS reference. This reference did yield some needed detail, but it was still not clear what sorts of textbook items would be categorized specifically as examples of the given cognitive processes by the authors of the manuscript. For example, was the activity of equation solving categorized under the process, “Solve Routine Problems,” within the Applying category or was it rather considered under the process, “Compute,” within the Knowing category? There is somewhat of a blurring between the classic categories of “knowing that” and “knowing how” with “compute” being included under Knowing and “solving” under Applying. The fact that the Applying category had such a high incidence (as shown in Figure 6) adds another reason for supplying more detail as to what kinds of items were considered to be included under Applying (as well as for the other two domains of Knowing and Reasoning). Furthermore, the process that is named “Recount” was not included in the TIMSS Advanced 2008 Assessment Framework. All of this leads to the suggestion that each of the cognitive processes that were listed in Figure 4, and which were used in the subsequent coding of the textbook items, be accompanied by at least one example of a textbook item that illustrates that particular cognitive process. While such information could be included in the given subsection, another possibility would be to add an Appendix, so as to not break up the rather tight flow and briskness of the manuscript’s narrative.

Author Response to Comment 2: Here there are two suggestions from Reviewer 1. (a)   We appreciate Reviewer 1’s concern about the brevity of Section 2.2 regarding the analytic framework for analyzing cognitive behavior across the six textbook series. We were deliberately brief, not wanting to duplicate methodological details discussed in [43], which we explain at the top of p. 5: “Complete methodological details, including our rationale for choices regarding study design, are provided in Huntley et al. [43].” However, in light of Reviewer 1’s comments, we think we may have been too brief in our explanation in Section 2.2 of the manuscript and therefore in the revised version have added Figures 4 and 6, together with text explaining use of the analytic taxonomies (pp. 7–8). (b)   Reviewer 1 is correct that the subprocess “recount” does not appear in the Knowing domain of the cognitive behavior framework from the TIMSS Advanced 2008 Assessment Framework [44]. The subprocess “recount” emerged from our coding, and therefore we added it to the cognitive behavior analytic framework that we used in analyzing the textbooks. We have included a couple of sentences in the revised manuscript (p. 7, lines 246–250) to explain this, and thank Reviewer 1 for bringing our oversight to our attention.

 

Reviewer 2 Report

Comments and Suggestions for Authors

The manuscript presents an interesting analysis of algebra textbook problems using six textbooks and a large sampling of problems (63,174). The authors present the findings of this analysis in regard to the varying algebraic content in the context of supporting student learning. The authors contribute to the literature and offer something unique to the readership in their very comprehensive and deep analysis of the textbook problems considered for this study. According to the authors, this differs from previous studies, which looked at portions of algebra textbooks or only at the table of contents in algebra books. Overall, the study presents interesting findings an and provides an extensive literature review. It is well written with appropriate methodology, results, discussion, and conclusion. A recommendation is to add a section on how this analysis could shape future textbooks and algebra instruction to support student learning. While the literature review is comprehensive, another recommendation is to add a theoretical framework.

Author Response

 

Response to Comments from Reviewer 2

Thank you very much for taking the time to review this manuscript. Please find detailed responses to your comments below. The corresponding revisions and corrections are in the revised manuscript, in which track changes was utilized.

Reviewer 2 Comment 1: A recommendation is to add a section on how this analysis could shape future textbooks and algebra instruction to support student learning.

Author Response to Comment 1: Thank you for this suggestion. We have added a paragraph to the conclusion of the revised manuscript (p. 20, lines 760–767) that discusses how our analysis can shape future textbooks and algebra instruction to support student learning.

Reviewer 2 Comment 2: While the literature review is comprehensive, another recommendation is to add a theoretical framework.

Author Response to Comment 2: Thank you for pointing this out. We thought we had been clear in our communicating that Figure 1 serves as our theoretical framework. We have revised the wording on p. 2, line 120 of the revised manuscript to explicitly state this.