A Content Analysis of the Algebra Strand of Six Commercially Available U.S. High School Textbook Series

Abstract

Algebra as a school subject is ill defined. Students experience algebra quite differently depending on the perspective of algebra taken by authors of the textbooks from which they learn. Through a content analysis of problems (n = 63,174) in the narrative and homework sections of six high school mathematics textbook series published in the U.S., we acquired systematic and reliable information about the algebra strand (i.e., symbolic algebra and functions) of each textbook series. We introduce plots to show the density, distribution, and sequencing of content, and present analyses of data for cognitive behavior, real-world context, technology, and manipulatives. Feedback on this study from an author of each textbook series is shared, and findings are discussed in terms of students’ opportunities to learn.

1. Introduction

There is tremendous variation in students’ opportunities to learn the content and nature of algebraic activity in K-12 mathematics classes across educational jurisdictions in the United States (U.S.) and abroad [1]. Yerushalmy and Chazan [2] note that “School algebra is a complicated curricular arena to describe, one that is undergoing change” (p. 725). Kieran [3] stated that “Since the mid-1980s, the content of school algebra has been experiencing a tug of war between traditional and reformist views” (p. 709). Traditional (non-reform) approaches treat the concepts and skills of algebra in two separate year-long courses, usually separated by a year of deductive geometry. Textbooks aligned with this approach typically have a strong symbolic orientation. Although functions, along with their graphical, tabular, and letter-symbolic representations, are also included in textbooks with more traditional approaches, they are generally accorded a more minor role. In contrast to the traditional approach, reform approaches to algebra tend to give greater weight to functions, to various ways of representing functional situations, and to the solution of real-world problems using methods other than symbolic manipulation (e.g., using graphing calculators or computers). Textbooks aligned with this approach are often integrated, with the content strands of algebra, geometry, trigonometry, statistics, probability, and discrete mathematics appearing throughout each year of high school mathematics.

When asked, “What is algebra?” Hyman Bass [4] described both the state of the algebra landscape and the origins of the current “tug of war”:

The modern point of view … especially in the reform curricula has sort of taken the position [that] the world has changed a lot …. And so they’ve accordingly transformed the way they think of introducing algebra. So for a function, classically [it] was given by a formula, y = f (x), and then you’d try to graph it, etc. And now, a function is given by a collection of data points … you want to look for some pattern in them. So it’s a kind of an homage, a bending in the direction of analysis of data and trying to fit data …. So, it leads to very different curricular trajectories and various different things that get foregrounded and backgrounded …. If you think of algebra as sort of situated on a big landscape or a big mountain … there’s sort of different trails around the territory but they’re not highly connected with each other. And in particular, the set of skills and knowledge that they cultivate by the time kids leave high school are very different, depending on which of these approaches you take …. The curriculum people sort of took the liberty to redefine algebra. Basically in terms of curriculum, they’re actually talking about a different subject, to some extent, that doesn’t merge with classical algebra until probably post high school …. So instead of clarifying the language, and explain what the two things are that people are engaged in, they’re using the same name and doing a kind of colonial war about which one has the rightful claim to the territory …. It’s not at all clear that there’s a well-established consensus about which kind of treatment is most appropriate for all students.

This quote underscores the idea that algebra, as a school subject, is ill defined. Students experience algebra in quite different ways, depending on the perspective of the content taken by their teachers and the textbooks they use. It has been well established that textbooks are a strong determinant of what students have an opportunity to learn and what they learn [5,6,7,8,9,10,11]. Given the important role and influence of mathematics textbooks, together with lack of clarity as to the content of algebra in textbooks, it is important to understand what algebra is, as conveyed through textbooks, and thus, what students have an opportunity to learn.

The purpose of this research was to explore the content of the algebra strand of six high school textbook series (intended for students ages 15–18) in coherent, comprehensive, and commensurable ways. We defined the algebra strand to include symbolic algebra and functions. Three research questions guided this investigation:

• What algebra content, including the breadth, sequence, and depth of topics covered, do students have opportunities to learn?
• What sets of behaviors are expected of students as they engage with the content?
• To what extent are problems set in real-world contexts, and to what extent are tools (technology and manipulatives) required to solve problems?

We investigated these questions through rigorous application of coding taxonomies to all items from the narrative and homework sections of six different textbook series. In addition, we engaged an author of each of the textbook series in a set of tasks to introduce them to our research methods and confirm that our application of codes reflected the objectives embedded in their textbooks. The goal of this study was to acquire systematic and reliable information about how algebra is represented in high school mathematics textbooks published in the U.S. In the sections that follow, we provide some background and framing for this study, which includes a discussion of the role of algebra in the high school curriculum, a rationale for this study, and a summary of prior content analyses.

1.1. Centrality of Algebra in the School Curriculum

Algebra plays a central role in the school mathematics curriculum. U.S. national curricular guidelines recommend that algebraic thinking be included in all grades, K–8, and that high school students study algebra for at least one year [12]. Knowledge of algebra is considered a gateway to future success in postsecondary endeavors and is increasingly being required for high school graduation. Gamoran and Hannigan [13] presented evidence that all students benefit from learning algebra—among students with very low prior achievement, the benefits are somewhat smaller, but algebra is nonetheless worthwhile for all students. Moses and Cobb believe that “math literacy—and algebra in particular—is the key to the future of disenfranchised communities” [14] (p. 5).

The importance of learning algebra is underscored in numerous documents. In a report from the Mathematical Association of America, an argument was made that algebra is “necessary for everyone desirous of participating in our democracy” and that “the language of algebra has invaded virtually every discipline, including the social sciences and business” [15] (p. 6). In the report by the Committee on Prospering in the Global Economy of the 21st Century, the authors wrote that “students who choose not to or are unable to finish algebra 1 before 9th-grade—which is needed for them to proceed in high school to geometry, algebra 2, trigonometry, and precalculus—effectively shut themselves out of careers in the sciences” [16] (p. 102). As articulated in the report of the RAND Mathematics Study Panel [17], “We need systematic, reliable information on how algebra is actually represented in contemporary elementary and secondary curriculum materials, as designed and as enacted” (p. 49). Algebra is an area in critical need of further research.

1.2. Textbooks as Dominant Forces in the School Mathematics Curriculum

Textbooks and related ancillary materials used by teachers and students during instruction have a strong, perhaps unrivaled, influence on what and how mathematics is taught [18,19]. Reports from the National Assessment of Educational Progress indicate that mathematics teachers use textbooks as a major source for selecting content and instructional activities [20]. The report from the 2018 U.S. National Survey of Science and Mathematics Instruction states that “commercially published materials … exert substantial influence on instruction, from the frequency with which instruction is based on them to the ways teachers use them to plan for and organize instruction” [21] (p. 155). In this survey, it was found that 61% of U.S. high school mathematics teachers reported using commercially published textbooks as the basis of their instruction at least once a week. In addition to textbooks being used by teachers in planning and implementing lessons, textbooks serve as resources during teacher professional development and give policymakers insight into how recommendations from professional organizations and district/state mandates are implemented.

Figure 1 shows the theoretical framework that guided our work, adapted from Remillard and Heck [22], that situates the curriculum within a broad system of policy, design, and enactment that captures the relationships among different instantiations of the curriculum. The arrows in the diagram represent paths of likely influence as they are experienced in the U.S. system and illustrate that textbooks provide a critical link between the intended, enacted, and attained curriculum in school mathematics [23,24].

Valverde et al. stated that “perhaps only students and teachers themselves are a more ubiquitous element of schooling than textbooks. As such a central facet of schooling, understanding textbooks is essential to understanding the learning opportunities provided in educational systems around the world” [11] (p. 2). As a result, scholars have called for mathematics education researchers to focus more attention on textbook analyses [25,26,27,28,29,30,31]. This has indeed been happening—in 2013, Fan [32] noted that “school textbooks have received increasing attention in the international research community of mathematics education over the last decades” (p. 765).

1.3. Analyses of the Algebra Strand in High School Textbooks

Other researchers have examined the content of the algebra strand of high school mathematics textbooks. Sherman et al. examined the author-recommended homework assignments in 16 textbook series that are used in Algebra 1 courses [33]. They replicated the result obtained by Nathan et al. [34] that a symbol precedence view (in which symbolic problems precede verbal problems) is more prevalent in textbooks than a verbal precedence view (in which concepts are first embedded in verbal situations).

Another study of the algebra strand of high school mathematics textbooks was conducted as part of the Comparing Options in Secondary Mathematics: Investigating Curriculum (COSMIC) project [35,36]. These researchers conducted textbook analyses investigating how linear functions were treated in an integrated textbook (Core-Plus Mathematics Course 1) and contrasted this with the approach to linear functions as developed in textbooks having a subject-specific organization (using Algebra 1 textbooks by various publishers). Using a table of contents analysis, they documented the extent to which the topic was present in each textbook. They used these data to develop assessments that reflected students’ opportunities to learn linear functions using the different textbooks.

Two studies examined the content of algebra textbooks with the goal of informing adoption decisions. In one study, Project 2061 researchers at the American Association for the Advancement of Science [AAAS] worked with a team of teachers to analyze 12 Algebra 1 textbook series by comparing content and instruction related to specific learning goals from the domains of functions, operations, and variables [37]. The goals of this project were to assist adoption committees in making algebra textbook adoption decisions, help teachers revise existing materials to increase their effectiveness, guide developers in the creation of new materials, and contribute to the professional development of those who use their analysis procedure. In the other study, Rivers examined page 110 (or the nearest group of word problems) of five first-year algebra textbooks adopted for use in South Carolina in 1984, investigating the extent to which females and ethnic minorities are represented in word problems [38]. They also extended the research to examine the textbooks selected for the 1990 adoption process to investigate the extent to which changes had been made in response to the Standards [39].

The Mathematics Curriculum Study, conducted in conjunction with the 2005 U.S. National Assessment of Educational Progress High School Transcript Study, explored the relationship between course taking and achievement by examining the chapter review questions from approximately 120 Algebra 1, Geometry, and Integrated Mathematics textbooks to identify the mathematics topics covered and the complexity of the exercises (i.e., degree of cognitive challenge) [40]. Among the findings, researchers reported that core content made up about two-thirds of Algebra 1 courses, courses varied widely in the mathematics topics covered, and school course titles often overstated course content and challenge.

Across this collection of studies involving analyses of the algebra content strand within various textbooks, it is notable that the examination of textbooks by other authors was restricted to portions of textbooks and not entire textbooks. Some researchers’ methods of analysis entailed close examinations of specific problem sets [33,40] or pages [38]. Other researchers focused on examining topics in tables of contents [35,36] or particular learning goals [37]. Of the studies reviewed here, the Brown et al. study [40] included cognitive challenge, the Rivers study [38] focused on racial and gender representation, and the Sherman et al. [33] and Nathan et al. [34] studies focused on mathematical representations. This study reported here presents a novel methodological approach: (a) We examined the entire algebra strand within multiple textbook series through problem-by-problem analyses; and (b) we focused our analysis beyond content alone, including cognitive behavior and tools (technology and manipulatives). Moreover, this study is unprecedented in that an author of each textbook series was invited to comment on our methodology and findings relative to their textbook series.

2. Materials and Methods

Our research design was informed by existing methodological approaches to textbook analyses [36,37,41,42] and by the 2004 report from a committee of the U.S. National Research Council (NRC) [27]. The following section includes an outline of our research design and methodology. Complete methodological details, including our rationale for choices regarding study design, are provided in Huntley et al. [43].

2.1. Textbooks and Unit of Analysis

The teacher’s edition of six U.S. textbook series was analyzed (see Figure 2). The content of two textbook series was integrated: Core-Plus Mathematics Program (CPMP) [44] and Interactive Mathematics Program (IMP) [45,46,47]. Within these books, only units within the first three years that have a major focus on algebra were coded. These units were identified through discussions with the textbook series’ authors. The other four series that were analyzed include content that is subject-specific. Of these, three series underwent extensive field-testing during development: Center for Mathematics Education (CME) [48,49], Discovering Mathematics (DM) [50,51], and University of Chicago School Mathematics Program (UCSMP) [52,53], and one series was commercially generated (Glencoe) [54,55]. For these four subject-specific textbook series, the entire Algebra 1 (CME, Glencoe)/Algebra (DM, UCSMP) and Algebra 2 (CME, Glencoe)/Advanced Algebra (DM, UCSMP) books were coded.

The unit of analysis in this study was an item. We assigned a separate set of codes to every part of a multi-part question (e.g., 1a, 1b, 1c). Every textbook item in the narrative (excluding worked-out examples) and exercises (homework problems) of each series was coded. This approach to textbook analysis—in which every item was examined in both the narrative and exercises portions of the books—is unprecedented. This time and labor-intensive process enabled us to capture the systematic sequencing of content, cognitive behaviors, and tools that authors have built into their textbooks.

2.2. Analytic Frameworks

To code the mathematical content of the textbooks, we considered several frameworks, including the algebra component of the TIMSS Advanced Mathematics Framework [56], but decided to use the Survey of Enacted Curriculum [SEC] K–12 Mathematics Taxonomy [57] because it offered a more granular look at the content. In Figure 3, the 16 main categories of this taxonomy are shown, together with the sub-topics for basic algebra, advanced algebra, and functions. The algebra and advanced algebra content areas represent a symbolic approach to algebra.

We added extensive annotations to the coding taxonomies to illustrate how coders were to apply content codes to textbook items. The annotations consisted of specific mathematics problems (sometimes with illustrative examples from problems within the textbooks we were coding) and notes to coders (to facilitate consistent application of the content codes). For example, in Figure 4, we provide a portion of the original and annotated taxonomies corresponding to the content topic multi-step equations (code 507) within the algebra portion of the SEC Taxonomy [57]. Note that the content taxonomy is a coarse sieve. For example, with code 507 (multi-step equations), no distinction is made between evaluating, setting up, or solving multi-step equations.

The categories of cognitive processes from the TIMSS Advanced 2008 Assessment Framework [56] were used to code the cognitive behaviors expected of students as they engage with the mathematical content in the textbooks. As shown in Figure 5, this framework consists of three domains: knowing, applying, and reasoning. The cognitive process recount did not appear in the TIMSS Cognitive Behavior Taxonomy [56]. We added it to account for items that we were encountering in the textbooks that fell within the scope of the knowing domain but did not belong to the other subcategories (recall, recognize, compute, and retrieve).

In Figure 6, we provide an illustration of how the cognitive coding taxonomy [56] was used. In this example, there are problems that appear early in the UCSMP textbook series for each of the three cognitive domains: knowing, applying, and reasoning. The content code for each of these problems is 507 (multi-step equations).

In summary, the two primary curriculum variables that were examined in this study were content and cognitive behavior. The codes for content are on a numerical scale that has a partial ordering—the higher the code, the further in the school mathematics curriculum one expects to find the content. The cognitive behavior scale is a nominal scale in that the codes of knowing, applying, and reasoning are not ranked in a cognitive hierarchy. As a result of these choices, there were certain analyses that we could (and could not) perform with the data.

2.3. Coding Procedure

Five people who have strong mathematical content knowledge and experience in grades K–12 classrooms coded the textbook series—three university faculty and two high school classroom teachers. A coding manual was used in a 2.5-day coder training session that was held before coding commenced. Each coding team consisted of a teacher and either a university mathematics educator or mathematician. Each coder worked separately to apply codes to items in the narrative and homework sections of a textbook. A textbook item was assigned up to three content codes (each consisting of a three- or four-digit number) and a single cognitive code K (knowing), A (applying), or R (reasoning). In addition to content and cognitive behavior, binary codes were used to indicate real-world context, technology, and manipulatives. Every effort was made to retain the intention of the codes in the original SEC and TIMSS taxonomies. At the same time, the taxonomies were treated as living documents, with updates being made as new issues arose. Codes were entered into a spreadsheet (which we call a “coding template”). Through a process of negotiation, the coding pair came to an agreement on the set of codes to apply to each item. Consistency in coding the textbooks was maintained throughout the coding process through frequent conversations across coding teams. When necessary, portions of textbooks were recoded when a clarification was made to a taxonomy that changed the meaning of a code.

The software package Mathematica (versions 8 and 9) was used to tabulate and explore the data [58,59]. Descriptive rather than inferential statistics were used in our reporting of the data in the sections that follow.

3. Results

In this section, data that were gathered from coding all items from the narrative and homework sections within the algebra strand of six different textbook series are presented. Findings from the content analysis are presented first, followed by findings from the cognitive behavior analysis. We conclude with an analysis of the extent to which problems were set in a real-world context and required the use of tools (technology and manipulatives). Our goal here is to describe and summarize the data, not to make inferences or judgments.

3.1. Findings from the Content Analysis

The first research question was: What is the content, including the breadth, sequence, and depth of topics covered? In this section, we address this question by sharing three aspects of the content analysis.

3.1.1. Number of Items Coded and Emphasis on Algebra, Advanced Algebra, and Functions

Altogether, 63,174 items were coded across the six textbook series. Although our analysis focused on codes from the basic algebra, advanced algebra, and functions categories of the taxonomy, we did not restrict our coding to these three categories. The number of items that were coded in each textbook series is shown in

Table 1. Findings from the content analysis.

		Items Coded in the Algebra Strand
Textbook Series	Number of Items Coded	Algebra % (n)	Advanced Algebra % (n)	Functions (n)
CME	12,752	$35.3 \left(4501\right)$	$41.8 \left(5330\right)$	$33.7 \left(4297\right)$
CPMP	7546	$27.0$ (2037)	$27.8$ (2098)	$47.5$ (3584)
DM	10,446	$26.1$ (2726)	$23.9 \left(2496\right)$	$29.5$$\left(3082\right)$
GLENCOE	17,020	$41.2$ (7012)	$34.8 \left(5923\right)$	$21.3$ (3625)
IMP	1735	$39.2$ (680)	$32.6$ (566)	$34.2$ (593)
UCSMP	13,675	$35.5$ (4855)	$32.0$ (4376)	$27.0$ (3692)

. It may seem surprising that there is an order-of-magnitude difference between the number of items coded in the CPMP and IMP series and in the other four textbook series. We offer two explanations for this. First, recall that only selected units were coded from the first three years of the CPMP and IMP textbook series (those that the authors identified as having a major focus on algebra), whereas two entire books were coded for each of the other four series (CME, DM, Glencoe, and UCSMP). Second, the integrated textbook series (CPMP and IMP) are problem-based, with little or no narrative or explanatory text. By contrast, in the non-integrated series (CME, DM, Glencoe, and UCSMP), each section consists of narrative text that includes problems for students to solve, which we coded.

In addition to the number of items coded for each textbook, Table 1 also shows for each textbook series the percentage of items in which at least one content code from the algebra, advanced algebra, or functions portions of the taxonomy was used. In Table 1, the sum of each row does not add up to 100%. Recall that up to three content codes were assigned to each item, so an item could have been assigned a code from more than one of these three broad content domains (algebra, advanced algebra, functions) or none of these. According to the data in Table 1, the percentage of items coded as involving algebra ranged from 26.1% (DM) to 41.2% (Glencoe). The percentage of items coded as involving advanced algebra ranged from 23.9% (DM) to 41.8% (CME). The percentage of items coded as involving functions ranged from 21.3% (Glencoe) to 47.5% (CPMP).

These data provide a gross measure of the content emphasis within each textbook series, which is one portrayal of students’ opportunity to learn. The two books that were coded from the CME textbook series provides the greatest opportunity for students to learn about content in advanced algebra. Moreover, CME may offer the most opportunities for students to learn across algebra, advanced algebra, and functions, as indicated by the most double or triple coding across the three broad content domains: algebra, advanced algebra, and functions. As another example of how the data suggests students’ opportunity to learn, in the selected units of the CPMP textbook series that were coded, students have considerably more opportunity to learn about functions than they have an opportunity to learn about symbolic algebra (as indicated by the data in the columns labeled algebra and advanced algebra). Also, compared with the other five series that were examined, students using the DM textbooks have the fewest opportunities to learn about and practice problems involving symbolic algebra. In this series, 20.5% of the problems were not coded as belonging to the algebra, advanced algebra, or functions content domains.

3.1.2. Most Frequently Used Content Codes

Figure 7 contains a list of the ten most frequently used content codes for each textbook series, together with the percentage of items that were assigned that code. The data in Figure 7 span the entire 217 topic codes; that is, we did not limit these data to codes within the basic algebra, advanced algebra, and function categories of the content taxonomy.

Two content topics appear in the top ten list of all six textbook series that were coded: rate of change/slope/line and quadratic functions. In fact, the code for rate of change/slope/line is the first or second most frequently used content code in five of the six textbook series. Three content topics appeared in the top ten list of five of the six textbook series: systems of linear equations, operations on polynomials, and matrices and determinants. CPMP and IMP, the two textbook series with integrated content, both have three function topics in their respective lists of top ten content codes: linear functions, quadratic functions, and exponential functions.

3.1.3. Density, Distribution, and Sequencing of Content

To examine the density, distribution, and sequencing of content across the textbook series, plots of topic codes as a function of “time” were generated (see Appendix and Appendix B). In these plots, time is represented as the sequentially coded items in the textbook. Each dot represents a content code applied to an item within the textbook. The diagrams use transparency; that is, lone dots are faint, and as they overlap, the image gets darker. It is not intended for one to see clearly the position of each dot. Rather, the aim is for these graphs to reveal differences in density, distribution, and sequencing of content topics.

Timeline dot plots for the four subject-specific textbook series (CME, DM, Glencoe, UCSMP) are shown in Appendix A. Reading each plot from left to right, the content in the textbooks progresses from the first page to the last; that is, the dots are at the exact places where that content occurs in the textbook. The categories of content (listed in Figure 3) appear along the left-hand margin of the plots, beginning with number sense/properties/relationships at the bottom and progressing upwards to instructional technology. The dashed vertical lines in each timeline plot indicate the textbook within each series from which the data came. For example, the CME series has textbooks entitled Algebra 1 and Algebra 2. The part of the graph to the left of the vertical dashed line contains data from the Algebra 1 book, and the part to the right contains data from the Algebra 2 book.

Three general conclusions are shared here regarding the timeline plots for the subject-specific textbook series. First, there is a higher density of dots for the content category algebra in the portion of the plots corresponding to the Algebra 1 (Algebra) books, and the density of dots in this content category decreases as one progresses to the Algebra 2 (Advanced Algebra) books. Similarly, there is a lower density of dots for the content category advanced algebra in the portion of the plots corresponding to the Algebra 1 (Algebra) books, and the density of dots in this category increases as one progresses to the Algebra 2 (Advanced Algebra) books. A second conclusion from the timeline plots concerns the place within the series where content related to functions begins. For CME and UCSMP, functions are introduced approximately halfway into the first book, whereas for DM and Glencoe, functions are introduced early in the first book. For all four subject-specific series, content related to functions is emphasized more heavily in the second book in the series, as evidenced by the heavier concentration of dots corresponding to functions to the right of the vertical dashed line. The third conclusion from the timeline plots concerns the emphasis on other content areas within each series. For example, CME more heavily emphasizes number (especially early in the series) compared with the other series. In the DM series, there is more emphasis on statistics and data displays as compared to the other series, and the density of dots in the three broad content domains, algebra, advanced algebra, and functions, is less as compared to the other series. This is consistent with the data for DM shown in Table 1. Topics in measurement are prominent in CME, Glencoe, and UCSMP.

Timeline dot plots for the two integrated textbook series (CPMP, IMP) are shown in Appendix B. Because only the problems from units that the authors identified as having a major emphasis on algebra were coded, large swaths of these textbooks were not coded. The units that were coded are displayed in the timeline plots, using the same ordering of the units as in the textbooks themselves. (The units that were not coded are not represented in the plots.)

Three general conclusions are shared here regarding the timeline plots for the integrated textbook series. First, as with DM and Glencoe, the timeline plots for CPMP and IMP illustrate that the concept of functions is introduced early in the first unit of these series. Moreover, there is a high density of codes related to functions throughout these two series. This is consistent with findings shown in Table 1, which indicates that within the CPMP units that were coded, 47.5% of the items were coded as having content related to functions, and within the IMP units that were coded, 34.2% of the items were coded as having content related to functions. This is also consistent with the data presented in Figure 7, in which four of the top ten codes for each of these two series are related to functions. Second, the low density of dots in the timeline plot for IMP reinforces data provided in Table 1; namely, there are considerably fewer items in the IMP series as compared to the others. The third conclusion regarding the timeline plots for the integrated series is that in addition to functions, the content areas of algebra and advanced algebra are also heavily represented. As with the subject-specific textbook series, there is a higher density of dots for algebra earlier in these plots (and hence the textbook series) and a higher density of dots for advanced algebra later on.

3.2. Findings from the Cognitive Behavior Analysis

The second research question was: What sets of behaviors are expected of students as they engage with the content? Recall that each item in the textbooks was assigned a cognitive code from one of three domains: knowing (the facts, procedures, and concepts students need to know), applying (the ability of students to make use of this knowledge to select or create models and solve problems), and reasoning (the ability to use analytical skills, generalize, and apply mathematics to unfamiliar or complex contexts).

In Figure 8, bar graphs represent the percentage of items in the six textbook series that were coded as belonging to each cognitive domain. The salient point is not the actual percentages of items that fall into each cognitive domain but rather the pattern of differences in the data.

In all six textbook series, more than half the items were coded as involving the cognitive domain applying. Glencoe had the most items coded as falling into the knowing domain, and IMP had the fewest. CPMP and IMP had the highest proportion of items coded as involving the reasoning domain, and DM, Glencoe, and UCSMP had the fewest.

We view the graphs as useful visualizations of the relative proportion of items coded in each of the three cognitive domains. Looking across these data, certain textbook series provide a more balanced opportunity to learn across the three knowledge domains (CPMP, IMP, CME), whereas other series emphasize one or two knowledge domains over the other(s) (DM, Glencoe, UCSMP). The variation in these distributions of data conveys one characterization of how these cognitive domains are valued in each textbook series and thus provides a lens into the opportunities students are provided to experience the content.

The cognitive domains of knowing, applying, and reasoning are not ordered. Consistent with the view articulated in the TIMSS Advanced 2008 Assessment Framework [56] that “understanding a mathematics topic consists of having the ability to operate successfully in three cognitive domains” (p. 17), we believe that learning a concept involves students solving problems that correspond to all three cognitive domains—knowing, applying, and reasoning. As a result, we caution against trying to conclude that higher percentages of reasoning, applying, or knowing items are a priori better than another ordering of the percentages.

3.3. Other Findings

The third research question was: To what extent are real-world context and tools (technology and manipulatives) included in problems in each textbook series? Recall that the codes for each of these curriculum variables are binary—either the feature is present in the textbook item, or it is not. Data regarding real-world context and tools are reported in

Table 2. Percentage of items coded as being set in a real-world context or involving the use of tools (technology or manipulatives) for each textbook series.

	Real-World Context	Calculator or Computer	Manipulatives
CME	10.3	10.3	0.1
CPMP	44.9	22.2	0.3
DM	43.8	33.4	1.3
Glencoe	22.9	13.5	0.9
IMP	54.3	13.3	1.8
UCSMP	28.3	19.9	0.2

.

These data indicate that the extent of use of real-world contexts varies widely across the six textbook series that were coded. The algebra strand of the CME textbooks has the fewest items that are set in a real-world context, at 10.3%. The algebra strand of the Glencoe and UCSMP textbooks have 22.9% and 28.3% of items, respectively, set in a real-world context. The textbook series with the most items set in a real-world context are IMP, CPMP, and DM, which contain 54.3%, 44.9%, and 43.8%, respectively.

Compared with the percentage of items set in real-world contexts, the variation was smaller regarding the percentage of items that required the use of calculators or computers. The CME, IMP, and Glencoe textbooks contain the fewest items that required the use of calculators or computers, at 10.3%, 13.3%, and 13.5%, respectively. The DM, CPMP, and UCSMP textbooks contain the most items that required the use of calculators or computers, at 33.4%, 22.2%, and 19.9%.

Hands-on manipulatives were required for very few items in any of the six different textbook series. Across the series, the percentage of items that required the use of hands-on manipulatives ranged from 0.1% (CME) to 1.8% (IMP).

4. Feedback from Textbook Series Authors

We conducted independent analyses of the textbooks. At the same time, we considered it important for the authors of the textbooks to understand the taxonomies and how they were applied to their textbooks, and to confirm that our application of codes reflected the content and behavioral objectives embedded in their respective textbooks. Thus, we invited an author of each of the six textbook series to engage in a series of coding exercises. Our goals in doing so were threefold: (1) to familiarize authors with our coding procedures, (2) to illustrate how the data were generated by having them code selected portions of their textbooks, and (3) to provide us with feedback regarding our coding process and our application of codes. An author of each textbook series accepted our invitation to collaborate with us (remotely, via e-mail) and was paid a stipend for doing so. As outlined in Figure 9, each author was asked to complete four tasks. In the sections that follow, we focus our discussion on the authors’ written reflections (task 4). A comparative process of qualitative analysis was used to analyze the textbook authors’ responses to the reflection task.

4.1. Textbook Series Authors’ Views on the Coding Taxonomies

There was considerable agreement between the codes that we applied to items in the authors’ textbooks compared with the codes applied by the authors. Instances in which there were disagreements seemed to reflect the authors’ dissatisfaction with the coding taxonomies rather than with our coding. They articulated this in various ways. For example, one author said the following:

[I] did the best [I] could under the constraints of the allowable categories, but I have serious concerns about the categories themselves …. I found the categories quite unsatisfactory for describing the design of [my textbook] (or any curriculum, for that matter) … so many of the problems [that I coded in my textbook] are aimed at helping students develop the habit of abstracting a process from repeated numerical calculations. So, while a suite of problems may all look like computations, they are actually building to a punchline.

A different author expressed dissatisfaction with both the content and cognitive behavior taxonomies in the following way:

Given the tool used, K-12 Math Taxonomy, the codes applied were as accurate as they could be. However, this tool seems too simple and limiting …. For example, One-Step Equations (solving one-step linear equations OR setting up and solving one-step equations) are coded as if these two types of tasks are at the same level of complexity. Most students can solve questions like 5x − 3 = 7. Students can do the naked algebra. What they can’t do is generate this simple type of equation from a situation. Solving an equation is not the same as setting up and solving the equation. If that were true, most students wouldn’t dread the so-called word problems in a traditional algebra textbook. Another concern is that the codes and the domains can not and do not reflect the complexity of the task. For example [at this point the author provides a specific example from her/his textbook] this is the correct coding but this particular problem takes hours to solve and requires sophisticated ways of organizing the data, none of which you can pick up from a code and a domain. I realize … the categories of cognitive processes are supposed to help differentiate between knowing, applying, and reasoning with algebra. However, these two sets of tools can only give a glimpse of what the authors’ goals and purposes are for students [sic] understanding and learning of algebra.

Some of the authors noted that the taxonomies do not take into consideration how teachers use textbooks. One author stated the following:

One thorny issue here is that ostensibly routine curricular content can be taught richly in a way that fosters cognitive complexity and deep understanding, and, on the other hand, ostensibly rich curricular content can be taught in a way that routinizes it and makes it cognitively shallow. For example, we have had well-meaning teachers break down the [problems] into small-step worksheets so that they are easier for students (and teachers) but this then short circuits the thinking process and detracts from the deep-understanding goals. This can make the coding problematic …. Related to this is the issue of how much scaffolding is provided, how it’s provided, and when. This is an adjustment that is made on the fly by good teachers in the classroom, but is very difficult to craft when writing a textbook. And it can make the coding problematic …. [In our textbook we] state the problem in a more open-ended way in the stem, then provide “hints”, which are given as parts a-d of the problem. There are typically notes in the Teachers Edition that point out the options of less or more scaffolding. Of course, the students and teachers almost always go straight for the hints, parts a-d. However, if students did the problem in the less-scaffolded way, then the cognitive code becomes R, rather than K and A.

4.2. Textbook Series Authors’ Views on the Methodology

In addition to commenting on their perceived limitations of the coding taxonomies, several authors commented on the methodology of this study. One author expressed concern that not all of the units of integrated textbooks were coded:

It is very difficult to take an integrated curriculum where algebra is learned, developed, and used as a language over four years and compare it to a traditional algebra book. You would have to look at every problem in every unit [in an integrated program] over all four years to see all the ways algebra is learned and used.

Another author expressed concern regarding our decision not to code worked-out examples in the narrative portion of the textbooks:

I can understand the concept of NOT coding the worked examples in the text, but in each lesson of [my textbook] you will often find open-ended questions that are there to at least prompt some thinking, and at best open a class discussion …. These questions can pop up anywhere including in the “solution” to a worked example.

This particular author, who has a background in statistics, was also critical of our coding of every item in the books, thinking that our coding fewer items would give sufficient information, saying, “Coding a dozen randomly selected sections will give you the cognitive information you wish.” One author praised our coding of the narrative portion of lessons in the textbooks, saying the following:

The inclusion of examples from the narrative of a lesson as coding items is a good idea. It is more reflective of the intent of [my] textbook, since the … author assumes a student would read the lesson at some point.

A different author pondered our expanding this study:

It might be interesting to triangulate results from coding the student text, the teacher edition, and the provided assessment resources. What sort of content and cognitive behaviors are emphasized in the teacher edition? In the quizzes and tests, and other assessment resources? How do the expectations in the student edition, teacher edition, and assessments compare?

The various authors were not in agreement about whether the data we collected would accurately portray the material in their respective textbooks. An author said, “The process of coding content and cognitive behavior, applied consistently and accurately, can certainly be helpful in capturing what’s presented in textbooks and, to some extent, what’s expected of students.” Consistent with this view, another author said, “You are certainly getting a great deal of information about the book. And I believe that for the most part you will get quite accurate information.” However, this same author tempered this view by saying the following:

What make[s] one text different from another is the developed progression of thought as much as the topics covered and depth of each. So content and cognitive behavior I would say are complete and fairly accurate. But a good text also must put these pieces in an order that makes sense to students so that ideas accumulate to reinforce one-another and connect between. How can this be measured or judged?

Other authors did not share these beliefs that the data we had collected would accurately portray the fundamental nature of their textbooks. One author explained their thinking as follows:

I don’t think this process will capture the essence of the book’s content and cognitive behaviors of students as they engage with the content… The way you tackle a problem in [my textbook], [students are] taught to not take everything at face value. You need to understand why pi is pi, or why this formula is this formula, or why we are solving this problem this way. I’m not sure coding textbooks can capture this level of difference between textbooks. It is hard to capture how students are asked to create their own problems and discover relationships and generalizations.

Consistent with this view, the author of a different textbook said the following:

Coding a textbook cannot reflect the essence of the book’s content. How does one capture essence by coding? Any coding process is simply a compilation of check marks or instances of the inclusion of a content topic. Essence is much more encompassing than a quantity of check marks.

4.3. Advice and Other Comments from Textbook Series Authors

In their feedback to us, some of the authors made remarks about the vast amount of data we collected and provided advice, offered suggestions, and raised concerns about how to present it. One author wondered whether “any sort of training or other professional development [would be] needed or desired in order for someone to interpret coding results.” Two authors noted that they found the process of coding the lessons assigned from their respective textbooks to be tedious. One said, “Although I found this work tedious, I’m glad I had the opportunity to do this”, and another said, “The actual coding of [my] textbook was a tedious but enjoyable and reflective exercise.”

In reflecting on this research project, one author offered the following insight about how different views of algebra, learning, and schooling lead authors to develop quite different textbooks:

I think that there are very different beliefs about what algebra is, what is important for students to know and be able to do in algebra, what learning is, and what the purpose of school is. If you think the learning of algebra is primarily to pass on a certain body of mathematical knowledge, you develop a certain type of textbook. If you believe that students should learn to think and reason mathematically, and use algebra where appropriate, you develop a different type of textbook.

A summary comment by a different author offered praise: “[We] applaud the project team for attempting to analyze such a gargantuan set of data. We look forward to reading the analysis and conclusions.”

5. Discussion

We begin this section by reflecting on and responding to some of the textbook authors’ concerns. We next discuss our ideas for further investigations and conclude by revisiting ideas related to curricular choices and opportunity to learn.

5.1. Limitations of This Study

The textbook authors provided thoughtful feedback on many issues related to this study and articulated a number of limitations. Many of these limitations reflect budgetary constraints (e.g., our not coding all units of the integrated textbooks) and/or relevance to the research questions being investigated (e.g., our not triangulating results from coding the student text, teacher edition, and assessment resources).

Two comments from authors merit further discussion:

• “So while a suite of problems may all look like computations, they are actually building to a punchline;” and
• “What make[s] one text different from another is the developed progression of thought as much as the topics covered and depth of each … a good text also must put these pieces in an order that makes sense to students so that ideas accumulate to reinforce one-another and connect between.”

Indeed, the glue, the stuff that connects the pieces together to make a textbook coherent, is not captured very well through the use of the content and cognitive behavior taxonomies chosen for this study. These taxonomies are coarse sieves, which means that some things have necessarily fallen through the cracks. Related to this, the results of this study are a direct reflection of our unit of analysis, namely, our choice to code at the level of individual items in the textbooks. We think that these factors—the choice of taxonomies and unit of analysis—do not capture a book’s coherence very well, and we lend caution to overgeneralizing the findings. Each textbook series had quite a different feel to it as we coded it, yet sometimes the coding and resulting data did not reflect these differences. For example, there are remarkable similarities in the data and graphs we presented for the Glencoe and UCSMP textbook series, yet the authors of these books seem to have quite different philosophies and intentions for using them. The use of different taxonomies and/or a different unit of analysis would likely lead to different findings and patterns in the data.

We appreciate one author saying that this project does “not take into consideration how teachers use textbooks”. Indeed, this was beyond the scope of this project. However, some inferences about the intended use of the textbooks can be gleaned from the data. For example, as shown in Table 1, there is an order of magnitude difference in these data, suggesting that in one series, it is the authors’ intention that students are engaged in sustained investigation of a few problems (IMP) in contrast to teachers selecting from a myriad of problems in a text that has many more (Glencoe).

5.2. Constructing Mosaics: Ideas for Further Investigations

The goal of this project was to acquire systematic and reliable information about how algebra is represented in six high school mathematics textbook series in the U.S. We coded several dimensions of every item in the narrative and exercise portions of the six textbook series. These dimensions include content, cognitive behavior, problem context, and tools (technology and manipulatives). This comprehensive approach resulted in an enormous dataset. Each element in the dataset is like a tile in a mosaic, with each tile conveying a small piece of information. In a mosaic, a picture emerges when all the tiles are put together. Likewise, as we look across the various dimensions of coding for each textbook series, we see the portraits of algebra that emerge from the data. Further discussion concerning the mosaics is in Huntley et al. [43].

There is much more that our data can tell mathematics educators, policymakers, and researchers about the algebra strand of the high school mathematics curriculum in the U.S., as conveyed through the six different textbook series. Further analyses of the data can help to refine the portraits of algebra in the textbook series. For example, to obtain more fine-grained information, we might look for patterns within the data for each textbook series and examine in greater depth particular algebra topics across the different series. For instance, numerous items were coded as linear functions together with the cognitive process applying, and by drilling down more deeply into items assigned these codes, we can examine the different ways that this coding pair plays out in the textbooks, thereby extending the research by Chávez et al. [35,36]. Another potentially fruitful investigation involves examining the ways in which some textbook series link algebra to arithmetic (i.e., the number and operation domains of the SEC taxonomy [57]), whereas others link symbolic algebra to functions. We also wonder how the different components of a textbook series interact (e.g., whether the cognitive behaviors of problems are the same across the narrative portion of a book and the homework problems) and whether there is a particular sequence (progression) of items within the homework exercises and in what ways the narrative supports this. We can examine progressions of content with a smaller scope (e.g., investigate the mathematical storyline within a textbook series and see how it is related to the co-development of equation solving and learning of functions). We are interested in examining the technology codes more closely within each textbook series (e.g., Where in each textbook series is technology use the most prevalent? Does the use of technology [about 20% for three series and about 10% for three other series] match our expectations regarding what is advocated for in the field, and what might this say with respect to opportunities to learn?).

Another line of inquiry involves our collaborating with a biometrician. Making sense of our data, which involves taxonomic units across sections and chapters in textbooks, shares many similarities with data analysis issues faced by ecologists examining species abundances at various locations. Both situations involve massively multivariate data with typically very sparse matrices. In ecology, only a few species from a species list are found at a particular location. Similarly, in our situation, only a few topics from the content taxonomy are applied to problems in a chapter of a textbook. Highly specialized statistical software packages (e.g., Canoco 5.0 [60]) have been developed for ecologists, which we would like to use with the datasets we have generated. One issue this software may shed light on is determining whether coding fewer items will yield similar results. What we did in coding every item in the textbooks is the gold standard and is not feasible under typical circumstances (i.e., without funding and enormous amounts of time). We wish to investigate whether there is a way to determine how many and which sets of items to code within each chapter to minimize coder fatigue.

5.3. Curricular Choices and Opportunity to Learn

Revisiting the theoretical framework (Figure 1), we recognize that a textbook is just one factor that shapes students’ learning. We acknowledge the broad interacting web of factors that ultimately shape students’ understanding of algebra. Although the curriculum is instantiated through teacher and student engagement around various tools and artifacts, a textbook arguably provides a necessary and important starting point for making sense of the opportunities students have to learn mathematics. Indeed, as Porter said: “Knowing the content of the intended curriculum is important because the intended curriculum is the content target for the enacted curriculum” [61] (p. 141).

Textbook authors are influenced by their own deeply held beliefs about what features (curriculum variables) positively affect student learning. In this way, textbook writers’ values and perspectives on school mathematics guide their writing and provide specific opportunities for student learning. For example, in discussing the design principles of CME, Cuoco outlines the role of technical fluency, saying, “… reasoning about calculations in abstract symbol systems is useful [emphasis in original] … in the CME Project, we invite students to become fluent in algebraic calculations so that they can reason about them” [62] (p. 124). Our data are consistent with this perspective. As reported in Table 2, 10.3% of the items in the algebra strand of the CME textbook series are set in a real-world context. By contrast, 44.9% of problems in the CPMP units that have a major emphasis on algebra (as defined by the textbook authors) were coded as being set in a real-world context. This is consistent with one of the central tenets of the approach to algebra in CPMP: “The primary role of algebra at the school level is to provide effective models of numerical patterns and quantitative relations—in pure mathematics and in the many applications of mathematics in which numerical data are important” [63] (pp. 330–331).

As illustrated in these examples, based on textbook authors’ values and perspectives on school mathematics, authors make reasoned decisions about specific content and processes to include and emphasize in their books. Each set of authors has a set of best bets about what curriculum variables will positively impact student learning. If teachers use textbooks as the authors intend, then these best bets translate into opportunities for student learning. In this way, understanding the content of textbooks is essential to understanding the learning opportunities provided to students. As Hy Bass remarked, answering the question “What is algebra?” is more than a matter of content and cognitive behaviors; it is also a statement of beliefs and values. Representing textbook series through descriptive statistics and visual depictions such as those offered in this research offers readers a snapshot of those values.

6. Conclusions

Through this project, we acquired systematic and detailed information about how algebra is represented in six high school mathematics textbook series in the U.S., and we have provided concrete portrayals of algebra in the textbooks. By looking at several curricular variables—content, cognitive behavior, real-world context, technology, and manipulatives—we were able to capture robust characterizations, beyond broad brushstrokes, of what algebra is in each textbook series. Our choice of analytic frameworks provided one way, but certainly not the only way, to characterize algebra in the textbook series.

As discussed in Huntley et al. [43], a major contribution of this study involves the methodology. Authors of prior content analyses of the algebra strand of textbooks have looked only at portions of textbooks or tables of contents. Although we acknowledge the limitations of our study, our research contributes unprecedented findings on the algebra strand of textbooks across integrated and traditional course sequences by taking a comprehensive and systematic approach to data collection and analysis.

Another contribution of this study involves the display of data in timeline plots, which is novel in capturing visualizations of how content is sequenced across units and textbooks. Such analyses help to tell a story of not only what content is introduced but also in what sequence and with what density. We can imagine how such techniques of visualizing large data sets could be applied in other domains of research (e.g., learning progressions across high school courses) and practice (e.g., trends in students’ outcomes on standardized test measures).

Our methodology, namely, problem-by-problem analysis of all items within a textbook series, is the gold standard and cannot be replicated for every new textbook series that enters the market. However, one thing we learned is that our analysis confirms the statements of beliefs and values that some textbook authors have communicated in publications external to their textbooks. For this reason, we recommend that authors of future textbooks explicitly communicate their perspectives on how students learn algebra and their decisions about the order and emphasis of specific topics to help inform decision-makers, including teachers, who are tasked with textbook adoption decisions.

We reiterate the well-established axiom that students learn what they have an opportunity to learn. Our study sheds new light on the opportunities students are afforded to learn algebra when using specific textbooks but leaves open fundamental questions about what understanding and skill in algebra is most important for students to acquire from their school mathematics experience, and with what balance and sequencing of content and cognitive behavior.

Around the world, textbooks are important resources for teaching and learning mathematics. They play an integral role in defining mathematics as a school subject and shape the learning opportunities for students by teachers. As stated by Fan, “school textbooks have received increasing attention in the international research community of mathematics education over the last decades [yet this] field of research is still at an early stage of development” [32] (p. 765). Building on this study, together with a sustained inquiry into the data that were generated, will allow for more complete portraits of algebra to emerge in the intended U.S. mathematics curriculum.