Strengthening STEM Teaching in Rural, Indigenous-Serving Schools through Long-Term, Culturally Responsive Professional Development

Abstract

This paper describes a particular teacher professional development model offered in schools on and bordering the Navajo Nation in the southwestern United States. The Diné Institute for Navajo Nation Educators (DINÉ) offers professional development across all content areas and grade levels, but here we focus specifically on our work in science, technology, engineering, and mathematics (STEM) content areas. Our work is situated explicitly within the literatures on Indigenous education, Native Nation Building, and culturally responsive schooling, but we also draw broadly on research in STEM education and teacher professional development. The research question explored in this paper is: To what extent and in what ways do teachers in the DINÉ develop STEM curriculum units that evidence culturally responsive principles and STEM education best practices? We share findings from three cohorts of teachers in the DINÉ’s STEM-focused professional development seminars. Teacher-authored curriculum units developed in the DINÉ were analyzed with two specific protocols: the CRAIS Tool, and the SCOOP notebook. Finally, we look closely at the curriculum units written by a single teacher in the DINÉ across the three years in order to get a clearer understanding of the nuances and richness of the findings and themes reported from the aggregate data.

1. Background and Purpose

Teacher quality is the most important school-based factor impacting student success in schools [1,2]. This is not to discount the myriad of other factors that play a role in students’ schooling experiences, but it does highlight the critical need to support teacher growth and efficacy. Given the persistent structural inequities that adversely impact Indigenous youth across the U.S., improving teacher quality in Native-serving schools is an important strategy for increasing the educational attainment of youth who have long been ill-served in our schools. The well-known challenges to teacher retention are particularly acute in rural communities and in STEM content areas, so this adds to the urgency of efforts to support teachers in these areas. The Diné Institute for Navajo Nation Educators (DINÉ) aims to do this by engaging teachers in robust, long-term professional development opportunities that honor their expertise and challenge them to improve their practice. By working alongside university faculty who are content experts, teachers in the DINÉ can improve their own content knowledge and build a curriculum that is culturally responsive and particularly meaningful for youth across the Navajo Nation.

The research reported here is from a 4-year, NSF-funded exploratory study of a teacher professional development program called the Diné Institute for Navajo Nation Educators (DINÉ). The DINÉ is a partnership between Northern Arizona University (NAU) and Navajo schools, aimed at strengthening teaching in schools serving DINÉ and other Indigenous students. The Navajo Nation spans 27,000 square miles in the states of Arizona, New Mexico, and Utah. Approximately 333,000 people are enrolled tribal members, and roughly half of them reside within the Navajo Nation. The DINÉ was developed with guidance and support from the Yale National Initiative© (YNI), which began over 40 years ago as a partnership between Yale University and the New Haven Public Schools to strengthen teaching in that community. YNI expanded to a national initiative and Navajo teachers began participating in this program in 2011. In addition to supporting individual teachers through professional development at Yale University, YNI also encourages teachers to build partnerships in their home communities in order to establish local institutes that are modeled after the YNI approach to professional development. Navajo teachers reached out to Northern Arizona University in 2016 to request the university’s participation in the development of a program that would provide culturally responsive professional development to teachers from the Navajo Nation. After a year of collaborative visioning and planning, the DINÉ was launched as the first local institute through YNI to partner with a Native Nation and the first to serve rural communities.

The DINÉ professional development model emphasizes (1) multi-grade and cross-content-area collaboration among teachers, (2) teacher-developed instructional units, and (3) culturally responsive approaches to teaching and learning. This approach is especially important for teacher professional development efforts in Native-serving schools. “Because of the rural context and large geographic distances between schools and communities on the Navajo Nation, teachers rarely have access to professional development, and what they do receive is generally district-led, short-term, and not content-specific [3,4,5]” [6] (p. 325). The DINÉ model addresses this challenge by supporting teachers in the development of self-authored instructional units that are aligned to state content standards and DINÉ cultural standards. Overall, the DINÉ advances a collaborative, systems change approach by shifting the standard practices around both teacher professional development and curriculum delivery in schools on the DINÉ Nation. The DINÉ supports the creation, utilization, and institutionalization of a curriculum that is uniquely teacher-developed, culturally responsive, and academically rigorous. Shifting standard practices around professional development and curriculum delivery is critical for building resilience among both teachers and students on the DINÉ Nation.

While the DINÉ offers professional development across all content areas, this paper focuses specifically on our work in science, technology, engineering, and mathematics (STEM) content areas. We provide an overview of our professional development model, and we share findings from three cohorts of teachers in STEM-focused professional development seminars. In our model, the term “seminar” is used to describe the entire 8-month engagement and participation in a specific content area topic led by a specific university faculty member serving as the content expert. We focus especially on conclusions drawn from our use of the SCOOP Notebook rubric and the CRAIS Tool to analyze 35 teacher-authored STEM curriculum units produced in our program between 2019 and 2021.

Teachers in the DINÉ select a single theme-based seminar group in which to participate for the 8-month program. During the 2019, 2020, and 2021 program, five STEM seminar options were offered:

• The Human Body: Marvels of Physics, Chemistry and Biology working together! (6 teachers in 2019)
• Clean Air and Water on the Navajo Nation (7 teachers in 2019)
• Unpacking Place Value (7 teachers in 2020)
• Patterns, Relations, and Functions (5 teachers in 2021)
• Forests and Climate Change (10 teachers in 2021).

Each seminar was led by a single faculty member with relevant expertise in the content area, and teachers worked with this faculty member and their teacher colleagues for 8 months. During this time, teachers read extensively, attended lectures, and engaged in collaborative learning with the group.

Table 1. Program Structure and Key Milestones.

When?	What?	How?
Mid-March	Participants receive syllabus and reading assignments	Email
Late March	Orientation session led by DINÉ Director	Zoom
Early April	Seminar meeting	Zoom
April through June	Reading period: Participants read assigned readings and self-selected readings on topic of their choosing	On your own time
Late April	Saturday Seminar Meeting, all day	In person: NAU
Early May	Participants and Seminar Leader hold individual meetings to discuss personal reading list	Zoom
Mid-May	Topic and personal reading list due	Email
Early June	Evening Seminar Meeting, 2.5 h	Zoom
Mid-June	Prospectus due	Email
Late June	Summer residency, all day/overnight for 10 consecutive days	In person: NAU
Late June	Context & Rationale sections due	Email
Mid-July	Participants check-in with Seminar Leader	Zoom
Late July	Evening Seminar Meeting, 2.5 h	Zoom
Early August	First draft of curriculum unit due	Email
Mid-August	Saturday Seminar Meeting, all day	In person: Navajo Nation
Late August	Evening Seminar Meeting, 2.5 h	Zoom
Early September	Second draft of curriculum unit due	Email
Mid-September	Saturday Seminar Meeting, all day	In person: Navajo Nation
Mid-October	Final curriculum unit due	Email
October through December	Teach curriculum unit in your classroom	In your classroom
Early December	Showcase & Open House Event, all day	In person: NAU

below highlights the typical meeting schedule and due dates for the DINÉ.

The program culminates with each teacher writing a curriculum unit on a topic of their choosing within the general theme of the seminar. The final teacher-authored curriculum units are between 10–20 pages, and each include the sections outlined in

Table 2. Curriculum Unit Sections.

Context	This is a brief description of the community, school, grade level, content area, and any other pertinent characteristics of the class(es) for which the curriculum unit is developed.
Rationale	This is the teacher’s reasons for creating this particular curriculum unit, including why and how the topic under study is relevant and meaningful for the community.
Topic Summary	This is a clear statement of the subject matter the unit covers, including essential background ideas and/or concepts.
Teaching Strategies	This is a unified, coherent teaching plan for the topic of the unit. This describes what the teacher generally plans to do to engage students in the topic.
Classroom Activities	This provides three more detailed examples of actual teaching methods or lesson plans describing what the students will do as they engage the topic of the curriculum unit.
Student Assessment Plan	This describes how the teacher will assess student learning of the curriculum unit’s content.
Standards Alignment	This section lists the particular state curriculum standards and DINÉ standards the curriculum unit addresses.
Resources	This is a list of resources the teacher has reviewed and used, and/or recommends in their curriculum unit.

below:

The DINÉ curriculum units are not like traditional lesson plans or curriculum units. Instead, they are written in narrative form, and they focus heavily on articulating the topic of study, its importance to the community where the teacher works, and the relevance of the topic to Navajo culture and ways of knowing. All of the teacher-authored curriculum units are published on the DINÉ website and intended for other educators to access and use.

2. Related Research and Theoretical Framework

Our work is situated explicitly within the literatures on Indigenous education, Native Nation Building, and culturally responsive schooling, but we also draw broadly on research in STEM education and teacher professional development.

As we have explained in other publications, the DINÉ model embodies many best practices of teacher professional development: it focuses on content knowledge, is long-term, engages active learning strategies, and is aligned to local and state standards [7,8,9,10,11,12]. High teacher turnover is a barrier to maximizing the impacts of PD [13], and teacher turnover is an especially significant problem across Indian Country. But we also know that collaborative approaches are particularly well-suited for Indigenous contexts [14,15,16], and that a culturally responsive curriculum produces more engagement and learning [17]. There is a dearth of published research on STEM professional development in Indigenous-serving schools, so our research begins to fill this critical need. There are two notable exceptions that our work is in conversation with. First, Chinn [18] reported on a STEM professional development program where teachers were supported in writing a curriculum unit that centers around Native Hawaiian knowledge. Second, Kern, Honwad, and McLain [19] reported on a 3-year climate science professional development experience with teachers in Native-serving schools and noted that teachers in their study struggled to bring Indigenous knowledge into their science classrooms because they were not confident in this area, did not have knowledge of the Indigenous community, and did not have relationships built with people who could share Indigenous frameworks and knowledges with them. They also found that although the teachers in their program felt supported by the program leaders and staff, they did not feel supported within their schools and districts to deliver the content and employ the various instructional approaches they learned in the professional development.

There is a growing body of literature on the importance and integration of culturally responsive approaches to STEM teaching and learning specifically [20,21,22,23,24,25,26]. At the same time, the Framework for K-12 Science Education [27] promoted the idea of organizing school science content around disciplinary core ideas. Organizing the content around disciplinary core ideas provides the opportunity for deeper exploration of key disciplinary ideas and the development of meaningful understanding that is both flexible and coherent, in contrast to superficial and disconnected knowledge of the isolated facts of science, as has been the case with more traditional instruction. The need to develop a coherent understanding of key scientific ideas while making them culturally relevant for the students calls for a special kind of professional learning design for teachers that addresses these two needs in tandem. The DINÉ is designed to accomplish this by providing professional development seminars focused on STEM content, delivered by university-level content experts and aimed at guiding teachers in integrating that content into culturally responsive curriculum units for use in their own classrooms.

3. Methods and Data Sources

Our project overall uses a collective case study design informed by Critical Indigenous Research Methodologies to address the following research questions:

• To what extent and in what ways does the DINÉ professional development model impact teachers’ STEM curriculum development and instructional practice in Native-serving schools?
• To what extent and in what ways are culturally responsive approaches to STEM curriculum development and instructional practice engaged by teachers in the DINÉ?

However, this paper specifically reports only on the analysis of the teacher-authored curriculum units written in STEM-related seminars. The research question explored in this paper is: To what extent and in what ways do teachers in the DINÉ develop STEM curriculum units that evidence culturally responsive principles and STEM education best practices? We used two specific protocols to assess these units.

The first protocol is called the SCOOP Notebook rubric. The Scoop Notebook is a protocol for teacher-generated artifacts that combine features of portfolios and self-report in order to investigate teacher practice [24]. In its full form, each teacher collects and submits artifacts, reflections, and videos of their teaching over a 5-day period, and the researchers rate the data along 10 dimensions of STEM instruction derived from National Science Education Standards [28]. The protocol has been field-tested and validated. For our purposes in this paper, we use either the Science Rubric or the Math Rubric to assess the teacher-authored curriculum units written in the DINÉ program. The decision about which to use was determined by the focus of the seminar; some seminars were explicitly math-focused and taught by a Mathematics faculty member and others had a broader science focus and were taught by faculty in various science disciplines.

The SCOOP Science Rubric includes these domains:

(1)

Grouping: The extent to which the teacher organizes the series of lessons to use groups to work on scientific tasks that are directly related to the scientific goals of the lesson, and to enable students to together to complete these activities. Active teacher role in facilitating groups is not necessary. Examples include student-led facilitation and group engagement; teacher is there to support but not facilitate.

(2)

Structure of Lessons: The extent to which the series of lessons is organized to be conceptually coherent such that activities are related scientifically and builds on one another in a logical manner.

(3)

“Hands-On”: The extent to which students participate in activities that allow them to physically engage with scientific phenomena by handling materials and scientific equipment.

(4)

Use of Scientific Resources: The extent to which a variety of scientific resources (e.g., computer software, internet resources, video materials, laboratory equipment and supplies, scientific tools, print materials) permeate the learning environment and are integral to the series of lessons. These resources could be handled by the teacher and/or the students, but the lesson is meant to engage all students. By variety, we mean different types of resources OR variety within a type of scientific resource.

(5)

Cognitive Depth: Cognitive depth refers to a focus on the central concepts or “big ideas” of the discipline, generalization from specific instances to larger concepts, and connections and relationships among science concepts. This dimension considers two aspects of cognitive depth: lesson design and teacher enactment. That is, it considers the extent to which lesson design focuses on achieving cognitive depth and the extent to which the teacher consistently promotes cognitive depth.

(6)

Scientific Discourse Community: The extent to which the classroom social norms foster a sense of community in which students feel free to express their scientific ideas honestly and openly. The extent to which the teacher and students “talk science,” and students are expected to communicate their scientific thinking clearly to their peers and teacher, both orally and in writing, using the language of science.

(7)

Explanation & Justification: The extent to which the teacher expects students to provide explanations/justifications, both orally and on written assignments.

(8)

Inquiry: The extent to which the series of lessons involves the students actively engaged in posing scientifically oriented questions, designing investigations, collecting evidence, analyzing data, and answering questions based on evidence.

(9)

Assessment: The extent to which the series of lessons includes a variety of formal and informal assessment strategies that measure student understanding of important scientific ideas and furnish useful information to both teachers and students (e.g., to inform instructional decision-making).

(10)

Connections & Applications: The extent to which the series of lessons helps students: connect science to their own experience and the world around them; apply science to real world contexts; or understand the role of science in society (e.g., how science can be used to inform social policy).

The SCOOP Math Rubric includes these domains:

(1)

Grouping: Teacher organizes mathematical tasks that are directly related to the mathematical goals of the lesson and enables students to work together to complete these activities. Examples include student-led facilitation and group engagement; teacher is there to support but not facilitate.

(2)

Structure of Lessons: Teacher organizes conceptually coherent activities related mathematically and builds on one another in a logical manner.

(3)

Multiple Representations: Teacher’s lessons promote the use of multiple representations (pictures, graphs, symbols, words) to illustrate ideas and concepts of mathematical representations in an appropriate manner.

(4)

Use of Mathematical Tools: Teacher’s lessons open opportunities for appropriate use of mathematical tools (calculators, compasses, protractors).

(5)

Cognitive Depth: Teacher’s lessons address the “big concepts” and build connections and relationships between them. Lesson design effectively promotes cognitive depth.

(6)

Mathematical Discourse Community: Classroom social norms foster a sense of community where students feel free to express their mathematical ideas honestly and openly. Students can clearly discuss orally and written mathematical concepts.

(7)

Explanation & Justification: Teacher expects students to provide explanations and/or justifications, both orally and in written assignments.

(8)

Problem Solving: Teacher activities enable students to identify, apply, and adapt a variety of strategies to solve complex problems and allow for multiple solutions.

(9)

Assessment: Teacher conducts formal and informal assessment strategies to measure understanding of important mathematical ideas and information.

(10)

Connections & Applications: Teacher’s series of lessons help students connect mathematics to their experience, the world around them, and other disciplines. Lessons help students apply mathematics in the real world and other disciplines.

Both the science and mathematics rubrics for the SCOOP Notebook use a 5-point scale where five indicates the best and most consistent representation of a specific domain and one indicates no representation of that domain.

The second protocol is called the Culturally Responsive Assessment of Indigenous Schooling (CRAIS) Tool. Our research team developed this tool to assess the integration of culturally responsive schooling principles specifically in/with schools serving Indigenous communities. The CRAIS Tool is published online and also described elsewhere [29], and we used it similarly to the SCOOP rubrics to assess the units. The CRAIS Tool includes five broad domains, with each having between four and six specific rubric items. The domains and rubric items are listed below.

Relationality, relationships, and communities

• Encourages students to understand themselves within broader communities
• Relationships within and among local/regional Indigenous community are understood and/or reflected
• Encourages students to build and sustain relationships
• Relationships within the classroom are strong

Representation of Indigenous peoples

• Indigenous people are represented as contemporary (not only historical)
• Indigenous people are represented as diverse (not a monolithic “they”)
• Local/regional Indigenous community is reflected
• Clear reference and/or integration of local/regional Indigenous context
• Recognition that local/regional Indigenous context is specific and unique, as are other contexts

Critical understandings of diversity, and specifically race

• Actively works to counter stereotypes of Indigenous people and/or communities
• Models critical thinking about historical narratives and contemporary status quo
• Encourages asking critically oriented questions about historical narratives and contemporary status quo
• Diverse narratives and perspectives are integrated

Indigenous knowledge systems and language

• Traditional and/or cultural knowledge is included
• Norms, values, traditions, and interests of local/regional Indigenous community are leveraged for learning opportunities
• Local/regional context is leveraged for learning opportunities
• Local Indigenous language(s) is valued
• Local Indigenous language(s) is integrated
• Academic language is built, but not at the expense of local Indigenous language(s)

Sociopolitical context and concepts, and specifically sovereignty, self-determination, and nationhood

• Recognition of Native Nations as governmental agencies
• Recognition of treaty rights and/or federal Indian law
• Students are encouraged to exercise self-determination and agency
• Communities are encouraged to exercise self-determination and agency

Items in the CRAIS Tool are scored on a 7-point scale with negative and positive values, as follows:

−3:

High degree of opposite of what the item represents.

−2:

Medium degree of opposite of what the item represents.

−1:

Low degree of opposite of what the item represents.

0:

Absence of what the item represents.

1:

Low degree of presence of what the item represents.

2:

Medium degree of presence of what the item represents.

3:

High degree of presence of what the item represents.

The CRIAS Tool also includes a category of N/A to indicate if the item was not applicable to the specific instructional unit. This is different from the rating of zero, where the item was applicable but not addressed within the instructional unit.

Our research team consisted of five members with diverse identities, experiences in Indigenous-serving schools, and areas of expertise. Four of the five are people of color, two of whom are Indigenous from different Native Nations. Two of the team members have direct experience as teachers and leaders in Indigenous-serving schools, two others have conducted a significant amount of scholarly work related to Indigenous communities and education, and the fifth team member has expertise in STEM education and pedagogy. In an effort to norm our use of the rubrics, each team member initially independently assessed the same three randomly selected units, and we discussed our scores and adjusted our understanding of the rating scale where necessary until we reached a consensus on the general definitions and ratings. With this shared understanding and norming complete, each of the 35 curriculum units was randomly assigned to three team members, who each independently assessed their assigned units using both the CRAIS and SCOOP rubrics. Scores were then compiled into a composite spreadsheet. We report on the numeric findings and collective analysis below.

4. Findings and Discussion

We begin this section by highlighting the key aggregate findings across the three years reported in this paper. We look first at the data from the SCOOP rubrics, and then at the CRAIS Tool data. We discuss key themes that were consistently evident in the data and highlight how these themes are central to culturally responsive STEM instruction with and in Indigenous-serving schools. Finally, we look closely at the curriculum units written by a single teacher in the DINÉ during 2019, 2020, and 2021. By zooming in on a single teacher’s work produced across three years of our program, we get a clearer understanding of the nuances and richness of the findings and themes reported from the aggregate data.

4.1. SCOOP Findings

We begin our discussion with analysis of the SCOOP rubrics.

Table 3. SCOOP Notebook Average Scores by Subject and Year.

Science Rubric Domain	Average Score in 2019	Average Score in 2021	Math Rubric Domain	Average Score in 2020	Average Score in 2021
(1) Grouping	2.42 *	3.33	(1) Grouping	3.76	3.53
(2) Structure of Lessons	2.39 *	4.07 *	(2) Structure of Lessons	4.14 *	4.13 *
(3) Hands-on	2.46 *	3.55 *	(3) Multiple Representations	4.19 *	4.20 *
(4) Use of Scientific Resources	1.92	3.43	(4) Use of Mathematical Tools	3.05	3.47
(5) Cognitive Depth	2.23	3.93 *	(5) Cognitive Depth	4.00 *	4.00 *
(6) Scientific Discourse Community	2.50 *	3.34	(6) Math Discourse Community	3.24	3.40
(7) Explanation & Justification	2.35	3.57 *	(7) Explanation & Justification	3.81	3.33
(8) Inquiry	2.15	3.13	(8) Problem Solving	3.86	3.73
(9) Assessment	1.96	3.33	(9) Assessment	4.14 *	4.00 *
(10) Connections & Applications	2.81 **	4.33 **	(10) Connections & Applications	4.19 **	4.60 **
(11) Overall	2.42 *	3.70 *	(11) Overall	4.05 *	4.00 *

** indicates the highest scoring domain; * indicates the domains scoring in the top half of all domains.

below shows the cumulative average scores for all teacher-authored curriculum units written each year. For the three years reported in this paper, there were science seminars offered in 2019 and 2021, and math seminars offered in 2020 and 2021.

Across all of the 35 teacher-authored math and science units, domain 10 almost always evidenced the highest ratings from each of the raters. Domain 10 is “Connections/Applications,” and it was consistently rated highest in the cumulative averages (i.e., across all raters) as well. This consistently highest domain is indicated in Table 3 with a double asterisk (**). The “Connection & Applications” domain is defined in the SCOOP Rubric as: “Teacher’s series of lessons help students connect mathematics/science to their experience, the world around them, and other disciplines. Lessons help students apply mathematics/science in the real world and other disciplines”. Given the focus and structure of the DINÉ curriculum units, domain 10 should indeed be one of the higher evidenced areas across the units. Our PD model centers culturally responsive principles, and the collaborative discussions in the seminars often focus on connecting content knowledge to local culture, place, history, and knowledge.

We were also curious if additional domains scored consistently higher than others on the SCOOP rubric. To determine this, we excluded domain 10 (because it is clearly the most consistently high) and looked at the top half (5 of the remaining 10) of the domains in each given year and content area. For Science in 2019, this meant the domains that scored at or above 2.42. For Science in 2021, this meant the domains that scored at or above 3.55. For Math in both 2020 and 2021, this meant the domains that scored at or above 4.00. The top five domains for each year and content area are noted in Table 3 by an asterisk (*). We define “consistently high” in the discussion below as those domains that are in the top half in both years analyzed.

Across the 2019 and 2021 science units, domains 2, 3, and 11 (Structure of Lessons, Hands-on, and Overall) had consistently high scores. There was some variability in other domains that were in the top half one year but not across both years; this variance may be due to the fact that a different faculty member taught the science seminar in each of these two years. Each faculty member’s approach and emphasis were likely unique. Also, every single domain in the science rubric has a higher cumulative average in 2021 compared to 2019. This suggests that the teachers collectively grew in their ability to do the things being assessed in those domains, hopefully as a result of the discussions and feedback during the seminar sessions. This may indicate an improvement in the quality of their instruction.

For the math units in 2020 and 2021, domains 2, 3, 5, 9, and 11 (Structure of Lessons, Multiple Representations, Cognitive Depth, Assessment, and Overall) consistently scored high. There was more consistency in the high scoring items across the two years of math seminars than there was across the two years of science seminars. This could be because the same faculty member led the math seminar in both years. At the same time, the scores on each domain did not change much from one year to the other. They were generally higher than the science scores but remained almost the same from the 2020 math seminar to the 2021 math seminar. Also of note is that there is not much consistency in the higher ranked domains between the science and math units; indeed, the only shared item besides connections and applications is Hands-on/Multiple Representations. This inconsistency could be due to the different seminar leaders or the different content focus, or most likely, a combination of both of these two variables.

4.2. CRAIS Tool Findings

We turn now to an overview of the CRAIS Tool ratings on the 35 science and math units. The first, second, and fourth domains (Relationality, Relationships, and Communities; Representation of Indigenous Peoples; and Indigenous Knowledge Systems and Language) were generally rated highest across the units, with composite domain scores at or above 1.60 every year.

Table 4. CRAIS Tool Average Scores by Year.

Domains and Corresponding Items	2019	2020	2021
Relationality, Relationships, and Communities:	2.00	2.00	1.60
Encourages students to understand themselves within broader communities	2.26	2.15	1.85
Relationships within and among local/regional Indigenous community are understood and/or reflected	1.66	1.64	1.39
Encourages students to build and sustain relationships	1.84	2.15	1.70
Relationships within the classroom are strong	2.21	1.98	1.42
Representation of Indigenous peoples:	2.04	2.28	1.78
Indigenous people are represented as contemporary (not only historical)	2.21	2.32	2.00
Indigenous people are represented as diverse (not a monolithic “they”)	1.82	1.58	1.29
Local/regional Indigenous community is reflected	2.32	2.53	2.17
Clear reference and/or integration of local/regional Indigenous context	1.95	2.74	2.16
Recognition that local/regional Indigenous context is specific and unique, as are other contexts	1.92	2.22	1.30
Critical understandings of diversity, and specifically race:	1.81	1.31	1.33
Actively works to counter stereotypes of Indigenous people and/or communities	1.24	0.39	0.79
Models critical thinking about historical narratives and contemporary status quo	2.18	1.66	1.49
Encourages asking critically oriented questions about historical narratives and contemporary status quo	1.89	1.33	1.55
Diverse narratives and perspectives are integrated	1.92	1.86	1.49
Indigenous knowledge systems and language:	1.80	2.10	1.64
Traditional and/or cultural knowledge is included	2.39	2.41	1.96
Norms, values, traditions, and interests of local/regional Indigenous community are leveraged for learning opportunities	2.16	2.47	2.08
Local/regional context is leveraged for learning opportunities	2.11	2.61	2.19
Local Indigenous language(s) is valued	1.34	1.93	1.41
Local Indigenous language(s) is integrated	1.39	1.48	0.98
Academic language is built, but not at the expense of local Indigenous language(s)	1.39	1.68	1.24
Sociopolitical context and concepts, and specifically sovereignty, self-determination, and nationhood:	1.40	1.23	1.05
Recognition of Native Nations as governmental agencies	1.32	0.99	0.89
Recognition of treaty rights and/or federal Indian law	1.11	0.89	0.73
Students are encouraged to exercise self-determination and agency	2.08	2.12	1.60
Communities are encouraged to exercise self-determination and agency	1.08	1.02	0.96
Average of all 23 items	1.817	1.833	1.506

below shows the cumulative average scores for each item as well as their collective domain on the CRAIS Tool.

The CRAIS Tool scores indicate no clear pattern of improvement from 2019 to 2021 for any of the five domains, nor is there a pattern of improvement in the overall average of all 23 items on the CRAIS Tool. It is worth noting that two of these three years (2020 and 2021) were affected by several restrictions in school operations due to COVID-19. As a result, the teachers’ ability to participate in the program, as well as to implement these principles in their instruction, was severely limited. The PD meetings were offered completely online in 2020 and 2021, and our participants experienced disproportionate levels of COVID exposure, illness, and community-wide restrictions. Even with these constraints, it is worth noting that all scores (both individual items and the collective domains) are in the positive half of the rating scale, indicating that all elements of culturally responsive schooling assessed by this tool were indeed implemented by teachers in the program. Further, any scores of two or higher indicate performance in the top third of the positive scale since the highest possible score on the scale is three.

5. Zooming in to Explore One Teacher’s Units

We will briefly highlight two science units and one math unit in order to nuance these numeric data. There were a handful of teachers who were in some combination of science and math seminars across the three years reported here, so we look at one of these teachers’ units to provide a point of comparison. For the purposes of this paper, we call this teacher Annie. Annie is a fifth-grade teacher in a public middle school that serves just over 500 students in a community located in the middle of the Navajo Nation. She is DINÉ and has been teaching for over 30 years. She is a teacher leader in the DINÉ and has been participating and providing leadership to the Institute since its inception. She teaches all subject areas in her fifth-grade class, and she often integrates the DINÉ language and culture throughout her teaching. Between 2019 and 2021, she participated in two science seminars (2019 and 2021) and one math seminar (2020).

Annie was in a seminar called Clean Air and Water in 2019, and her curriculum unit is titled “Water on the Colorado Plateau”. She briefly describes this curriculum unit in this way:

Water is everywhere. Above, below, and even in our bodies. Water is very important and scarce in the southwest United States and on some parts of the Colorado Plateau, which encompasses the Navajo Reservation. We know water is recyclable. That is, it goes through the water cycle process that allows all living beings to consume clean water to sustain them. In the water cycle, water in the liquid state forms rivers, streams, tributaries, and lakes as surface water. Some water will seep into the porous rock and soil and become underground water. The goal of this unit is to teach students about why clean water is important. My integrated curriculum unit embeds the DINÉ culture and language, and aligns to the state Social Studies, Science, and Language Arts Standards.

In 2020, Annie was in a seminar called Unpacking Place Value, and her curriculum unit is titled “Place Value and the Navajo Stick Game”. She describes her 2020 math unit in this way:

My curriculum unit is designed to teach students to learn, understand, and know how to use the place value system while playing the Navajo Stick Game. There are many concepts and skills when teaching the place value system, and it is very adaptable for the Navajo Stick game. Place value encompasses the four operations (addition, subtraction, division, and multiplication), decimals, fractions, estimations, and rounding, and more. Teachers need to understand and to be able to teach the skills to students thoroughly and efficiently when using the place value system. To explain it efficiently, teachers need to use the mechanical application, use different manipulatives, and stress the relationships between math skills because they are all interrelated and have connections to the place value system.

And in 2021, Annie was in a seminar called Forests and Climate Change. The title of her curriculum unit from this seminar is “The Upside Down Forest and Climate Change in the Betatakin Canyon”, and she describes her unit in this way:

My curriculum unit is about Betatakin Canyon, located on the Shonto Plateau. I want my students to know the importance and uniqueness of the canyon’s fauna and flora. Each aspect of the canyon has different variations of plant and animal species and I hope to help students appreciate how the canyon is unique. I also want to teach them some of the geology of Betatakin Canyon. I want my students to know how land connects living organisms and natural resources. These relationships are needed to keep all beings in balance. My fifth-grade students are at an age where they can comprehend the interconnections of the living earth. I want my students to know more about how climate change affects Betatakin Canyon.

Even in these brief summaries, it is clear that Annie leverages familiar places, ideas, concepts, and games to engage her students in STEM content learning. This is an important focus of the DINÉ, as we have discussed above and elsewhere [6,30]. Annie’s average scores on the CRAIS Tool are noted below in

Table 5. Annie’s Average Scores on CRAIS Tool Items by Years.

Domains and Corresponding Items	2019 Seminar: Clean Air and Water	2020 Seminar: Unpacking Place Value	2021 Seminar: Forests and Climate Change
Relationality, Relationships, and Communities:	2.25	1.33	1.83
Encourages students to understand themselves within broader communities	2.5	1	1.67
Relationships within and among local/regional Indigenous community are understood and/or reflected	2.5	1.5	1.33
Encourages students to build and sustain relationships	1.5	1.33	2.33
Relationships within the classroom are strong	2.5	1.5	2
Representation of Indigenous peoples:	2.60	2.13	1.60
Indigenous people are represented as contemporary (not only historical)	2.5	2	2
Indigenous people are represented as diverse (not a monolithic “they”)	2.5	1.33	0.67
Local/regional Indigenous community is reflected	2.5	2.33	2.33
Clear reference and/or integration of local/regional Indigenous context	3	2.67	2
Recognition that local/regional Indigenous context is specific and unique, as are other contexts	2.5	2.33	1
Critical understandings of diversity, and specifically race:	2.0	1.54	1.04
Actively works to counter stereotypes of Indigenous people and/or communities	1	0.5	0.5
Models critical thinking about historical narratives and contemporary status quo	2.5	2.33	1
Encourages asking critically oriented questions about historical narratives and contemporary status quo	2	1	1.67
Diverse narratives and perspectives are integrated	2.5	2.33	1
Indigenous knowledge systems and language:	2.5	2.22	1.79
Traditional and/or cultural knowledge is included	2.5	2.33	1.67
Norms, values, traditions, and interests of local/regional Indigenous community are leveraged for learning opportunities	2.5	1.67	1.67
Local/regional context is leveraged for learning opportunities	3	2.67	2.33
Local Indigenous language(s) is valued	2.5	2.33	2
Local Indigenous language(s) is integrated	2.5	2	1.33
Academic language is built, but not at the expense of local Indigenous language(s)	2	2.33	1.67
Sociopolitical context and concepts, and specifically sovereignty, self-determination, and nationhood:	1.13	0.59	0.92
Recognition of Native Nations as governmental agencies	1	0	0.5
Recognition of treaty rights and/or federal Indian law	1.5	0	0.5
Students are encouraged to exercise self-determination and agency	2	1.67	1.33
Communities are encouraged to exercise self-determination and agency	0	0.67	1.33
Average of all 23 items	2.15	1.64	1.47

.

Looking only at the two science units (2019 and 2021), the ratings are higher on all but two items in 2019. As noted in the discussion of the average scores across all 35 units, the impact of the pandemic in 2020 and 2021 cannot be underestimated. It is also worth noting that we generally see higher ratings on the CRAIS Tool in science units (vs. math units) and especially those taught by social scientists. In Annie’s case, her 2019 seminar was taught by a faculty member in Applied Indigenous Studies whose expertise lies in Indigenous Environmental Justice, so we might expect to see higher alignment to the principles of culturally responsive schooling. Some of the items on the math rubric rated quite high for Annie’s 2020 unit. Across all three years, five items were consistently rated at a two or higher. These include:

• Indigenous people are represented as contemporary (not only historical)
• Local/regional Indigenous community is reflected
• Clear reference and/or integration of local/regional Indigenous context
• Local/regional context is leveraged for learning opportunities
• Local Indigenous language(s) is valued.

These five items on the CRAIS tool convey a shared emphasis on the local place, people, and context. Annie’s long history and deep connections to the community where she teaches allow her to make three different STEM curriculum units highly relevant and linked to her students’ locality. However, the centering and leveraging of the local place, people, and context does not simply happen by default. Indeed, Annie has shared that the curriculum her school gives her is irrelevant, standardized, and disconnected to her students’ lives and communities. But being in the DINÉ each year means that Annie has a professional space to develop a culturally responsive curriculum. While there are a number of ways Annie’s curricula align to the principles of culturally responsive schooling, the consistently high degree to which she aligns to the five principles listed above are indicative of how she has effectively centered her own knowledge and experience in the development of the curriculum through the DINÉ.

We turn now to the SCOOP rubric, which has a scale of 1–5. Annie’s average scores on the SCOOP rubrics are noted below in

Table 6. Annie’s SCOOP Notebook Average Scores by Years and Subject.

Science Rubric Domain	2019 Seminar: Clean Air and Water	2021 Seminar: Forests and Climate Change	Math Rubric Domain	2020 Seminar: Unpacking Place Value
(1) Grouping	3.00	3.33	(1) Grouping	4.00
(2) Structure of Lessons	3.00	3.67 *	(2) Structure of Lessons	4.00
(3) Hands-on	4.00 *	3.00	(3) Multiple Representations	4.33 *
(4) Use of Scientific Resources	3.00	2.67	(4) Use of Mathematical Tools	3.33
(5) Cognitive Depth	4.00 *	4.00 *	(5) Cognitive Depth	4.00
(6) Scientific Discourse Community	4.00 *	3.67 *	(6) Math Discourse Community	2.67
(7) Explanation & Justification	4.00 *	3.33	(7) Explanation & Justification	4.33 *
(8) Inquiry	4.00 *	3.00	(8) Problem Solving	4.33 *
(9) Assessment	3.00	4.00 *	(9) Assessment	4.33 *
(10) Connections & Applications	5.00 **	4.33 **	(10) Connections & Applications	4.33 *
(11) Overall	4.00 *	3.67 *	(11) Overall	4.33 *

** indicates the highest scoring domain; * indicates the domains scoring in the top half of all domains.

.

Similar to what was observed in the cumulative averages, Annie also has consistently high scores in domain 10 (Connections & Applications). This is particularly noticeable in her two science units, where the only 5 is in domain 10 in her 2019 unit and her highest score in the 2021 unit is also in domain 10. Table 6 has the same asterisk indicators as Table 3—that is, the double asterisk (**) denotes the highest scoring domain, and the single asterisk (*) denotes the domains scoring in the top half for each year and content area. There is less variation within each year in Annie’s scores as compared to the average scores across all 35 curriculum units (i.e., Table 3). There are also fewer consistently high scoring domains among Annie’s scores as compared to the analysis of all 35 curriculum units. The “Overall” domain is the only one (aside from domain 10) that is in the top half across all three years and content areas.

6. Significance and Next Steps

By better understanding the specific nuances of how teachers did and did not integrate core principles of both STEM pedagogy and culturally responsive schooling, we can more effectively tailor the experience in the DINÉ seminars to ensure maximum benefit among teachers. Two examples of how we have evolved in our practice from 2019 to 2021 are worth sharing here. First, we place strong emphasis on offering seminars that derive from the stated interests of teachers in the program, and then we intentionally select faculty members who evidence strong collaborative, caring, and community-engaged practice. And second, we now intentionally share the CRAIS Tool and the SCOOP rubrics with teachers in the DINÉ and use them as prompts to open up conversation throughout the year about cultural responsiveness and STEM pedagogy. The DINÉ leadership team shared a collective commitment to continuous improvement that is driven, first and foremost, by the feedback we receive from the teacher participants, so changes like those noted above are likely to evolve each year. Of course, we are also rebuilding from the impacts of the pandemic. The DINÉ continued through the worst years of this public health crisis, and we learned a lot about how we can engage hybrid approaches to meetings and more flexibility in deadlines throughout the program. The cultivation of relationships remains a high priority for the DINÉ, especially because it is critical to overall teacher well-being, so we continue to think about how to best create spaces for mutually sustaining connections among the DINÉ teachers, faculty, and staff.

The significance of our work in the DINÉ is two-fold: making STEM instruction more rigorous in terms of the depth of content and making STEM instruction more culturally responsive so that Indigenous students can connect STEM learning to their cultural contexts. This dual significance is illustrated by Annie throughout her three curriculum units, but we share just one small example from her 2021 unit here, where she wrote:

When I was young, I went on horse rides with my father. One day my father said, “Let us ride up the side of the Mesa and visit my father at Big Mountain”. While riding along the trail, my father talked about the land and how it helps sustain all living beings. He talked of how an abundance of mule deer, elk, wolves, and mountain lions had once roamed here. He said, “I rarely see them now because of the coal mine and because more people have moved onto the land”. Later, he talked about plants like juniper and cedar trees, sagebrush, and shrub oak and how the animals and people share the land and the natural plants. It was enjoyable listening to my father talk about animals, plants, and land. While we strolled up the trail, I analyzed the natural scenery while my father spoke about particular areas close to where we lived. It was an excellent oral lesson from him.

Embedded in this memory is a rich collection of STEM-related content knowledge. Annie’s reference to various flora and fauna, as well as the importance of sustainable relationships, is captured in this short story. Annie also talks about analyzing the land around her, while simultaneously listening and learning with her father—both important teaching and learning skills for STEM students.

A key aspect of the DINÉ professional development model is teacher leadership, and Annie’s units often serve as an example to her teacher colleagues about the possibilities that open up when teachers are supported in the development of a culturally responsive, high-quality STEM curriculum. Although each DINÉ seminar is led by a university faculty member, and although the expertise that the faculty members bring to the seminars is an important element of teacher learning, what may be even more important is the capacity of the faculty member to cultivate and facilitate a learning space where teachers feel supported in centering their lived experience, cultural knowledge, and community relations in the curricula they develop. In other words, the expertise the teachers bring to the seminars is equally, if not more, important than the expertise the faculty member brings.

As we look ahead to future STEM professional development with teachers in rural, Indigenous-serving schools, we would like to learn more about how faculty members think about and carry out this important work in the DINÉ. We are also interested in learning if the patterns noted in the 2019, 2020, and 2021 curriculum units remain consistent across new themes in science and math seminars in the years to come.