Framework for Implementing Improvement Science in a School District to Support Institutionalized Improvement

Abstract

This paper presents a framework for implementing improvement science in a district or educational organization. Findings from interviews with 17 experienced educational improvers indicate critical elements for a school district to effectively support and build the capacity of individuals and teams situated at multiple levels in the district in organizing, implementing, and sustaining improvement science methods. Utilizing a systems-thinking approach, this study conceptualizes how to institutionalize improvement science across an educational organization. To institutionalize the use of improvement science, district leaders must center schools and integrate the work in existing structures, offer support, and create coaching, growth management, and knowledge management plans.

1. Introduction

Fields such as manufacturing and healthcare have used improvement science to make progress toward solving long-standing problems and challenges [1]. In 2015, Bryk and colleagues published Learning to Improve, resulting in the use of improvement science throughout the education sector to solve problems through asset-based, intentional, problem-focused, systems-focused, disciplined inquiry [2]. Since Bryk and colleagues published their influential work, several more texts (e.g., [3,4,5,6,7]) have become available for educational stakeholders, such as school and district leaders, to learn more about using improvement science approaches.

While still limited, researchers have demonstrated that improvement science can be used to address problems of practice in P-12 and higher education spaces, including community colleges [2,7,8,9,10]. Improvement science has the potential to be a powerful approach to systemic change in United States (U.S.) districts, which tend to be initiative-focused [2]. Approaches to education improvement for the last few decades have primarily focused on short-term initiatives followed by more short-term initiatives. Years without drastic improvement call for rethinking the initiative-focused approach. This study is based on the premise that continuous improvement, which focuses on slowing down to deeply understand the problem from multiple perspectives, planning deliberate action based on root causes, and monitoring change over time, might have the potential for drastic improvement. However, without understanding how to sustain improvement science by institutionalizing it into educational organizations, improvement science may just end up mimicking the existing initiatives improvement science practitioners and researchers seek to counter.

There is still a need to understand how the process is embraced, implemented, and sustained in traditional educational organizations such as districts and schools. Implementing improvement science includes challenges related to time, resources, will, and skill. Researchers surmise that networked improvement efforts may not take hold in organizations due to the resource demands in systemic and systematic improvements as well as the coordination fatigue that can emerge from the efforts of the educational professionals involved in the improvement work [11]. An understanding, therefore, of how to identify and effectively use resources in improvement work is critical to successfully establish the structures to support sustainable improvement efforts.

The work is about shifting ways of thinking, being, and doing (e.g., organizational change), which takes time [12]. Tichnor-Wagner et al. found that there needed to be both top-down and bottom-up learning in organizations to support improvement science work, specifically that schools need to be able to adapt changes to fit their contexts for the implementation to be successful [13]. Redding et al. found that learning begets learning [14]. Despite initial hesitation about learning across school teams, the more team members engaged in this work, the more they wanted to continue sharing and learning with each other. Research on improvement science that explored the implementation of plan-do-study-act (PDSA) cycles, a tool for disciplined inquiry, found that educators believed in the importance of PDSA cycles but struggled to find the time to implement and track the progress of their cycles [15]. Harrison and colleagues suggest that a district must create a new culture, capacities, and resources to allow for improvement in addition to an “interlocking network of supports and structures to succeed” [16] (p. 1). These are the types of challenges we hoped to explore and address with this study.

1.1. Problem and Purpose of the Study

The purpose of this study was to establish key factors that support how districts can implement improvement science in ways more likely to ensure institutionalization. Institutionalizing district reform—and improvement science specifically—has proven to be difficult, but possible, in the United States education system (e.g., [17,18]). This paper presents an implementation framework informed by qualitative interviews with experienced improvers to enact the improvement science process in school districts. Indeed, with the recent emergence of improvement science experts for education settings, the field is well positioned to examine how those in PK-12 educational systems experience the development and implementation of improvement science in their contexts, investigating the bridge from theory to practice. This study sought to answer the research question: According to experienced improvers in the field, how might districts organize, implement, and sustain an improvement science model in the school and district? The framework developed from these findings will be useful for researching, leading, and implementing an improvement science model in a school or district.

Russell et al. developed a framework for networked improvement community (NIC) initiation to bring “a promising set of ideas that is now attracting considerable attention” into “actual execution” [11] (p. 27). The present study builds on this emergent work by leveraging learning from a decade of networked improvement communities in education to present a framework for how to embed improvement science—which regularly incorporates NICs—into a school district or other educational organization. By learning from people involved in the “actual execution” of improvement science, we can understand how to set up the organizational conditions from the start of the improvement science journey to ensure institutionalized improvement. Institutionalized improvement involves leadership, organizational change, and enabling structures and conditions to ensure that the system is prepared to make improvement science and NICs a routine part of the culture of the district or other educational organization. In this study, we illustrate ways to support school districts in the organization, implementation, and sustaining of improvement science processes that result in the institutionalization of improvement. This is one framework that complements and extends the field’s current understanding of how to implement improvement science approaches into a complex system, such as a PK-12 school district. Some specific systems, structures, and organizational routines that emerged as marks of institutionalization in the results include purposeful scheduling of improvement work, professional development related to improvement, resources including financial support for improvement work, collaborative teams to collect and analyze data related to improvement focus areas, and the role of different teams including leadership, among other collaborative structures, communication, evaluation, coaching, and induction/onboarding.

1.2. Literature Review

What follows is a brief review of the literature on improvement science and organizational structures for improvement and change in districts. We then describe our methods for planning, conducting, and analyzing qualitative interviews. We also describe an emergent framework for consideration when engaging with improvement science that developed from the coding that includes organizing, implementing, and sustaining improvement. We conclude with implications for practice and a summary of the work.

1.2.1. Improvement Science

Improvement science is a methodology to “define problems, understand how the system produces the problems, identify changes to rectify the problems, test the efficacy of those changes, and spread the changes (if the change is indeed an improvement” [6] (p. 1). Improvement science processes resemble other educational improvement processes situated under the umbrella of continuous improvement, which has been used in school improvement for decades, such as action research, design-based implementation research (DBIR), and total quality management (TQM) [19]. The process, however, is distinct in meaningful ways. Improvement science encourages a mindset of slowing down to deeply and empathetically understand a problem present in a particular setting to create a plan for improvement (e.g., [20]). The problem is understood by analyzing local data to determine variation in outcomes and experiences by conducting root cause analyses of the problem, and most importantly, by focusing on the experiences of those closest to the problem [2,21]. Improvement is monitored through thoughtfully designed and implemented small-scale tests of change or plan-do-study-act (PDSA) cycles [2,22]. The ability of users of this process to solve intractable education problems is magnified when prioritizing the dismantling of persistent inequities in schools and classrooms and using improvement science approaches to systematically disrupt inequities through changing organizational routines, structures, processes, and mindsets [6,23]. Improvers ask three key questions referred to as the “model of improvement” [19]: what is the exact problem I am trying to solve; what change can I test to solve the problem; and how will I know that change led to improvement? (e.g., [6]). Moreover, in asking these questions, improvers also consider who is involved and who is impacted [2]. Through problem identification, understanding the problem, developing a theory of practice improvement, testing changes, and then scaling successful changes, improvers engage with the model of improvement questions to make meaningful, sustainable changes in their contexts [3,6].

1.2.2. Supporting Multi-Level Organizational Change in School Districts

Educational change involves attention to the multiple levels of the district as an organization. Indeed, improvement science embraces the appreciation of the system, captured in one of the six key principles of improvement, which recommends improvers “see the system that produces current outcomes” [2] (p. 14). In the present study, we take a general view of “what counts” as a system. Systems can be considered organization levels (e.g., classroom, school, district) and systems may also refer to the processes and routines of education-centric practices (e.g., referral for advanced placement courses).

A system, in systems theory, “consists of interrelated parts that work together or in some fashion and impact each other in a process” [24] (p. 806), and systems views or approaches are “seeing and thinking that honor profound inter-connectedness, that nothing exists separately” [25] (p. 570). More broadly, in education literature, systems thinking has been used to understand organizational change. For example, Senge (2020) writes the following:

Systemic change is deeply personal and inherently collective. For example, while personal learning and development is essential, our work has long focused on working teams as a key first step that embeds individual learning within a larger social reality. Regardless of whether these teams are formally designated as such, like a school district management team, or informal, they are groups of people who need one another to get something done. Teams, in turn, nest both within formal organizational structures and informal networks of collaboration. At all these levels—individual, team, institution, and larger networks—deep change unfolds along both tangible and intangible dimensions.

[25] (p. 59)

A systems approach must attend to the various levels from the individual to the team to the organization to ensure that the interconnectedness and uniqueness of each level are attended to in the framework.

1.2.3. Districts as Change Organizations

District structures vary by geographic diversity, region, size, and other factors. However, commonly there are district leaders such as superintendents and chief academic officers who work closely with school boards, central office, and school-level leaders to ensure a common vision and mission, lead instruction, and manage operations [26]. In the organizational system of a school district, central office leaders are charged with financial and managerial concerns to ensure the organization is running smoothly [27,28]. District policy demands can ensure support for schools as:

Central office administrators work closely with their schools to build school-level capacity for high-quality teaching and learning and to use their experience as school assistance providers and other evidence to guide central office decisions in ways that promise to seed and grow such teaching and learning in schools districtwide.

[27] (p. 628)

Given their place in the system, central office leaders are well positioned to support schools in their instructional and improvement goals.

There is research to suggest that educational changes need to be embedded (e.g., part of the organization’s practice, routines, and norms) to be institutionalized and must be institutionalized to be sustained [29,30,31]. Researchers outlined ways district (or system) leaders can intentionally support and enhance equitable teaching and learning endeavors of school-based leaders and teachers (e.g., [18,27,32,33,34]). One way is for district leaders to take a learning organization approach to their work. As a learning organization, district-level leaders engage in continuous improvement work and are open to change [32,35]. District reform is often fraught and slow due to the complexity of the organizations and their predilection to be initiative-focused, even in smaller districts. Improvement science and other continuous improvement models [29,36] present a promising way to engage in district or school-level reform.

Evidence from the research also cautions educational leaders of the challenges of not only supporting and implementing change but also practices for sustaining change. “Sustaining change means integrating it into the larger systems of the organization” [37] (p. 777). Fullan (2009), for example, describes how change is predicated on three sets of core leadership practices, including setting directions, developing people, and redesigning the organization [38]. When redesigning the organization for improvement, systems and structures guide work processes and help support sustained change efforts. This is important as changes in practice can revert [18] during transitions when a key change agent turns over [36] or if the change is not viewed as worthy or connected to the overall purpose [38].

There is evidence that organizational routines allow practices to remain consistent and withstand changes such as turnover, while also allowing for the implementation of improvement [39,40,41]. With the goal of institutionalization, organizational routines help reduce complexity and bolster the types of structures and supports that sustain change. These organizational routines consist of repetitive (albeit flexible) actions and processes, completed by people who are influenced by their agency and the contexts in which they are operating [39]. Routines can convey priorities and build coherence, operate as feedback loops, and be tools for teams as they engage in professional practice [41] (p. 631). Institutionalizing organizational routines is important because routines serve as stabilizing forces and routes for improvement within organizations [39,40,41].

One important norm for district change is to ensure that changes to the system do not fall squarely on the shoulders of an individual or a small team. Creating a community of improvers will help changes persist even in times of shifting leadership or organizational and environmental stress. Examples of structures and supports that can help solidify new changes into workplace norms by supporting improvement include coherence making, strong relationships and trust, identification and elimination of barriers to collaboration and sharing learning, seeking diversity of perspectives, increasing collective capacity in the system, and connecting change to a moral purpose [38].

1.2.4. Collaborative Structures

One of the core principles and routines of improvement science is learning across collaborative groups. Improvement science research demonstrates the important role of collaborative networks in supporting improvement work [30,42]. Collaborative networks can be organized in different ways such as networked improvement communities (NICs) as a primary structure [2,9]. While similar, albeit distinct to a professional learning community (PLC) or communities of practice (COPs), NICs are teams of improvers sharing their learning with one another through coordinated means organized by a centralized hub. While there is a growing body of research centered on multiple aspects of improvement science both within and outside of networked improvement communities (NICs) (e.g., [43]), NICs are often part and parcel of improvement science. The studies on NICs are often focused on how to use NICs to sustain change ideas and packages (e.g., [42]). Much of the literature on the implementation of this type of disciplined inquiry focuses on scaling up change ideas throughout a district, often using NICs in research–practice partnerships (see [13,14,15,22,30,44]).

NICs interact with different levels of a system (e.g., school and district levels) and have characteristics including the following:

A well-specified common aim, guided by a deep understanding of the problem, the system that produces it, and a shared working theory to improve it; disciplined by the methods of improvement research to develop, test, and refine interventions, and organized to accelerate the diffusion of these interventions out into the field and support their effective integration into varied educational contexts.

[2] (p. 144)

Several publications (e.g., [9,11]) document how NICs develop and operate in educational spaces. In theory, because NICs are systematically sharing and learning across the network under the guidance of a hub, their improvement efforts are realized quickly. This is because all individuals and/or teams in a NIC do not have to learn each lesson from their own experience. Instead, teams can build off the previous learning from other teams. For example, in a network of four improvement teams, if team one learned that the change idea they tried and tested was not impactful and they share that information—including the conditions surrounding both the change idea implementation and testing—with teams two, three, and four, those teams may determine they do not need to try that change idea and can focus on a different test of change.

Research suggests that due to their disciplined structure, NICs are an effective way to accelerate learning [2,45], but it is essential for those engaged in improvement science and forming NICs to be intentional in their NIC development process. There is a need for a hub or similar centralized organizational leadership structure to create and support the enabling organizational conditions necessary for the effective sharing of information across a NIC. Russell and colleagues’ NIC development framework suggested that a successful NIC needs an initiation team, a common purpose, and utilization of research methods in improvement work with an organizational routine of capturing data while also developing a strong network culture with effective leadership [11]. In addition to the formation of a NIC initiation team, Russell and colleagues’ framework has three associated layers of NIC development: (1) the inner layer consists of learning how to carry out improvement science by using improvement science methods, developing a theory of practice improvement, and building a measurement and analytic structure for the NIC; (2) the next layer of the framework includes leadership, organization, and operation of the NIC; and (3) the external layer includes securing human, social, and material resources for the NIC. When operating in a district, organizations must also consider how to prepare for the NIC structure to flourish within the system as a whole [11].

Research like the work of Russell and colleagues [11] and Bryk and colleagues [2] reveals the complexity and multi-faceted nature of improvement as they include structural and leadership elements in forming this collaborative structure [2,11]. The present study includes NIC development but also focuses on implementing improvement approaches (e.g., understanding the problem and PDSA cycles) across and within systems (within a NIC structure or outside of a NIC structure) including elements of coaching, onboarding, and capacity building to reveal the complexity of the accompanying efforts of multi-level organizational changes for improvement in a school district.

In the NIC development framework, Russell and colleagues discussed using a measurement and analytics infrastructure, which is essential to networked learning throughout the NIC. A knowledge management plan would include this data infrastructure [40]. However, it would also include ways of moving distilled knowledge through the whole organization in a way that supports other types of existing learning. Becerra-Fernandez and Sabherwal [42] described a knowledge management system as ‘mature’ when it has four overlapping processes: knowledge generation and knowledge conciliation and capture, which are directly connected to the measurement and analytics infrastructure, and knowledge sharing and application, which include processes for communicating knowledge with others throughout the organization and applying knowledge to decision-making and improving practice more broadly [46]. Building from the implementation practice of learning through improving, a knowledge management plan allows for generation, consolidation, sharing, and application to occur through an understanding of current work processes, strong routines for generating and capturing learning, cross-network or cross-organizational routines for sharing knowledge, and context-specific knowledge about implementation that considers norms, mindsets, dispositions, and other necessary factors for application [47].

2. Materials and Methods

This study examined the following research question: according to experienced improvers in the field, how might districts organize, implement, and sustain an improvement science model at the school and district? To answer this question, this qualitative exploratory study was designed to develop an initial framework based on the insights of experienced educational improvers. Frequently, school districts start engaging in improvement science in partnership with universities or external providers or through involvement in district-led, grant-funded initiatives. In districts, there are individuals, both internally and externally, who are responsible for the content of the professional learning necessary to engage in improvement science, individuals who may have expertise or knowledge about the problem or problems being addressed, and individuals who must create the capacity and infrastructure for the work to be carried out [9,10,48]. These might all be different individuals or the same individuals with multiple areas of expertise. Often a team of district-based and external coaches and experts, sometimes from a university, work together and coordinate the efforts necessary for implementation. The participants in this study are adopters of improvement science in education who have served as coaches, district leaders, hub leaders, and researchers. The experiences of these improvers, gathered via interviews, were leveraged to co-construct an improvement framework to organize, implement, and sustain improvement science to address pressing problems of practice that are leading to inequitable experiences for P-12 students.

This study was designed as part of a research–practice partnership to support the organization, implementation, and institutionalization of improvement science in a large, public, urban P-12 school district and is part of a larger study. The district was in the second year of implementing a NIC focused on improving equitable math outcomes for students of color, students with disabilities, English Learners (ELs), and other marginalized groups of students. The NIC initiative was supported by a five-year grant and would eventually include all 38 high schools in the district that had a common goal or aim statement and common drivers (i.e., main areas connected to the problem of practice that need to be addressed to improve the problem). The schools were testing (i.e., piloting) change ideas through PDSA cycles, collecting data, and sharing their learnings across the schools in their NIC. The study sample was determined by the goals of the improvement work in the district work: to create a learning trajectory and an implementation framework for sustained organizational growth for one school district. To help us develop the learning trajectory and framework—and because the district was only in its second year of networked improvement—we sought out advice and insights from experienced educational improvers in the educational and improvement science fields who had multiple years of experience across the United States (e.g., other districts, organizations).

2.1. Sample and Data Collection

This study employed a purposeful sample followed by a snowball sampling technique (see [49]). We selected participants, with the help of leading organizations of improvement science, based on their familiarity with and knowledge of existing improvement work, NICs, and stories of success (see [50]). An organization at the forefront of improvement science implementation provided us with an initial list of potential participants. The researchers then invited the suggested participants to an interview (see

Table 1. Study participants.

Participant Pseudonym	Affiliation/Role	Years Engaged in IS	Interview Format
Denny	University professor, continuous improvement researcher	10+	Individual interview
Jacob	Non-profit leader, NIC leader, researcher, coach	10+	Group interview of two
Suzy	Non-profit leader, NIC leader, researcher, coach	10+	Group interview of two
Amanda	Non-profit leader, NIC leader, researcher, coach	10+	Individual interview
Jack	Non-profit leader, NIC leader, researcher, coach	10+	Group interview of two
Sarah	Non-profit leader, NIC leader, researcher, coach	10+	Group interview of two
Ericka	NIC leader, coach	10+	Individual interview
Irene	NIC leader, coach	10+	Group interview of two
Karla	NIC leader, coach	10+	Group interview of two
Jamie	NIC leader, coach	10+	Individual interview
Penelope	District A continuous improvement leader, non-profit leader	10+	Individual interview
Barry	District B continuous improvement leader, NIC leader, university leader	10+	Individual interview
Randy	District B continuous improvement leader, NIC leader	10+	Individual interview
Jocelyn	District C continuous improvement leader	5+	Group interview of three
Angela	District C continuous improvement leader	5+	Group interview of three
Steve	District C continuous improvement leader	5+	Group interview of three
Stacie	District D continuous improvement leader	5+	Individual interview

). The interviewees included seven hub and NIC improvement leaders from external organizations (i.e., not the school district of focus), eight experienced district/K–12 leaders, one university researcher who studies NICs, and one improvement professional from outside education. It is worth mentioning that this sample included six white men, seven white women, three Asian women, and one Black female. Although we sought a diverse sample, nationally, the work thus far has been disproportionately led by white improvers as reflected in the sample.

After reviewing the literature, the research team developed a semi-structured qualitative interview protocol (see Appendix A) and conducted 12 individual and group interviews with 17 participants [51]. Two members of the research team conducted each interview. When multiple people were interviewed at the same time, they were from the same organizational setting and had similar roles (e.g., same office in a school district; project leads for the same NIC). The interviews included questions about how one progresses from being a novice to developing` to expert improver; how to build leadership and organizational capacity for improvement; how to start the work in a new setting; and how to sustain the work. Several interviewees shared guiding documents from their own work and contexts to illustrate their responses, and we included those documents in the data collection and analysis.

2.2. Data Analysis

The interviews were loaded into NVivo 12 and open-coded for emergent codes using a line-by-line coding technique (see [52]). We did not begin with any a priori codes since this study was exploratory in nature. We generated codes from the data (e.g., de-emphasize tools, deep bench of improvers, focus on equity). After generating an initial set of codes, they were categorized into key factors for developing, implementing, and sustaining improvement. The relationships between and among the themes were then explored, resulting in the framework shared in the findings. The analysis was developed through an iterative, member-checking process that included gaining feedback from members of the organization on the findings and sharing the results with the district members. The researchers ensured credibility and trustworthiness through a triangulation of data from participants with similar roles or in similar settings for employing improvement science. All findings presented in this paper were supported by multiple interviewees and reflect a consensus among the improvers with whom we spoke. Representative quotes are shared below as illustrative examples to describe and explicate the themes and we provided participants with pseudonyms.

3. Results

In this section, we describe the framework developed for organizing, implementing, and sustaining improvement science for institutionalizing organizational change in a school district. To institutionalize improvement science into school districts as organizations, leaders must pay attention to implementing the plan and spreading both the improvement science process and knowledge gained from the process throughout the system. This institutionalization is not something that happens overnight. As Jack, who works for an external organization and is a leader of multiple NICs, pointed out, “If you’re not starting out thinking of this as at least a 10-year journey, you’re probably not understanding what the work is”. One reason why this journey is so long is that improvement science is a “set of tools, a set of methods. It’s a philosophy, but it’s also a set of mindsets and there’s a paradigm shift” (Karla, NIC leader and coach). This work may be carried out in non-linear phases and will likely adapt over time.

3.1. Overview of the Framework

There are two main aspects to organizing improvement science to support organizational change. First, NIC facilitators and district leaders must center the process around the work of schools and integrate the process into existing work. Secondly, the work should be structured for bottom-up decision-making with top-down support, finding the delicate balance between the two.

The participants suggested that implementing the work includes four key aspects. The first is that the improvers should learn about the improvement science, try it, and learn more through their engagement in the process. Second, the work needs to focus on the ‘why’ and be in service of meeting goals that increase equity and social justice for staff, students, and their families. The third aspect of implementation is to keep it simple: simple language, simple tools, and simple routines. Fourth, during implementation, the learning within the work should be differentiated by a person’s professional role. To sustain the work, leaders must develop a three-part plan that includes a coaching plan, a growth management and succession plan, and a knowledge management plan.

Figure 1 illustrates the Framework for Organizing, Implementing, and Sustaining Institutionalized Improvement Science that we developed from these data. Organizing the work is at the top because it must be considered prior to implementation across an organization. These organizing factors create the culture and capacity for implementation. Sustaining the work through planning is at the bottom because it undergirds the implementation process. These plans may change over time based on the level of implementation. The implementation practices are influenced by how the work is organized and by the plan for sustaining the work. We discuss each framework category next using the study’s data to support the framework elements.

3.2. Organizing the Work

The improvement science process will not become part of the system without a deliberate and clear approach to organizing the work to ensure structures and practices to support the implementation of improvement science. There are factors related to organizing the work that ensure institutionalization. These include centering on and integrating with schools and structuring for bottom-up decision-making with top-down support.

3.3. Centered on and Integrated with Schools

As the part of the organization most directly connected to students and student outcomes, participants suggested that the school should be at the center of a model for organizing support with support from the district level. Although improvement science can be enacted at all levels of a district, including as a method for district-level improvement, the goal of the process should directly connect to schools and their student-centered needs. In our interviews, Jamie talked about the district as a “layer of support for the schools”. She discussed how, in many districts, schools feel like they are supporting the district when the work should be focused on supporting schools in their improvement work. By asking schools to meet expectations of district mandates, the focus becomes on compliance with district needs rather than authentically supporting schools in meeting their needs.

An important aspect of this support mentioned by participants is that district leaders need to stop asking too much of school-based educators. Participants felt that if improvement science work is to be successful, expectations for improvement cannot be added to an already robust set of professional practices without removing other things or without integrating the process into existing structures and practices. Schools will not be able to adopt a continuous improvement approach or disrupt inequitable routines if the system (district) does not support them in doing so (beyond espoused theories). Suzy, an expert in leading and coaching improvement science referred to these organizational routines as systems of support: “I think you need to get district leaders to start… developing more coordinated systems of support around whatever the priority area is it you’ve chosen”. Corroborating Jamie’s earlier point about expectations of work, one participant, Randy, who runs NICs within a school, linked this to an opportunity cost:

I think it’s that you can’t be additive in education without subtracting things… So, if you’re asking them to collect data in the classroom, what are you taking away? And if you’re not taking away something, then that’s your responsibility as the asker.

When organizing for implementation, expectations of responsibilities like Jamie and Randy noted need to be considered alongside a realistic timetable for adoption. It is important to phase in the work and initiate only a few individual teams or NICs simultaneously as part of an organizational-wide effort. Going deep in one space will result in learning that can inform a long-term plan for expansion and growth from that original space—like the ideas behind starting small (e.g., with PDSA cycles). Amanda, a long-time improvement coach, warned, “I would not do it so big that you couldn’t actually support the teams”. She went on to suggest, “I would give them access to somebody who’s a more expert coach. And then when they learn how to support those teams, then I can start to expand”.

Finally, the participants shared that educational leaders need to dedicate time and space to this work. Amanda explained, “I would definitely give [educators] space and time in the workday, so it takes some time off their plate”. Without these, which ties back to how the work is being organized, the work will not integrate or embed. Suzy described, “Teachers have the mindset [of improvers], now they just need to be given the space and the context in which to be able to have that learning, like a sandbox to learn”. It is important to note Amanda’s asset-based view of educators in her above quote. She recognized teachers as naturally poised to be improvers. Leaders who recognize and harness this potential with necessary organizational support structures (e.g., time, space, and a scaling plan) will better position the organization for successful improvement work. This point also connects to the consideration of building capacity for improvement. The work of organizing for improvement is about developing capacity, as explained by Jamie:

I feel like there has to be a commitment to a multiple-year capacity building… I think of the places where I know it is. It gets embedded into the ethos of their system… They have key behavior indicators [of an] improver mindset, which is built into professional learning and evaluation. So, I think it has to be integrated into the system. I think that just these one-off projects are not going to make it sustainable in a school or in a district over time.

One way to structure the work is to create a formal role either inside or outside the organization. Karla, an improvement science researcher and NIC leader, talked about the need to create routines, which many participants also mentioned. As part of those routines and “in order for a collective of individuals to be doing improvement work together”, there must be dedicated personnel. Participants shared that formalizing roles, such as design team leaders and improvement facilitators, will support the implementation of the complex work.

Another suggestion mentioned by participants was to adapt improvement science to existing structures. They suggested embedding this work in data team meetings, instructional leadership teams, or in structures for other groups doing inquiry work and/or reviewing data like professional learning communities (PLCs) or communities of practice (COPs). Randy suggested that this organizational routine can be realized by asking questions such as:

What are some of the asks that we’re having for teachers? Can we use improvement on our PLC structures? Can we use improvement in our [professional development]? What are ways that you can infiltrate current systems or current structures and use improvement as the ways you do things in those structures? That’s how you integrate.

As much as possible, participants felt it was advantageous to build off existing routines for inquiry and data use.

3.4. Balance of Top-Down and Bottom-Up Support

The second consideration for organizing mentioned by participants is that the district must lead with a balance of top-down and bottom-up support to allow for the necessary school-level autonomy while providing the infrastructure and parameters that invite success. Simultaneously, educators appreciated some parameters for how the process could be embedded in daily work and the autonomy to make site-level decisions. For instance, participants shared that the district should create structures for reporting progress and learning and ensure that it is integrated into the existing work of leaders and leadership teams. The district should also create structures for supporting schools to focus on improvement. Amanda, a long-time improvement coach, recommended the following:

The change has to sort of build, but then it needs to be integrated into your system so, it becomes the default way of working. So, it doesn’t need people to hold it up anymore. That’s the idea. So, in a school, so you’re constantly from the beginning, the leader is setting that up from the beginning. Right from the get-go is setting it up and making the connections and keeping people along other people’s journeys.

The district or hub team should be integrating this improvement science mindset and practices into the system through organizational routines. However, participants recognized the balance that this requires. Irene, who has led multiple NICs, stated:

One thing that I think is really hard is learning how to strike the balance between being extremely user-centered and not making the entire process completely democratic. Being user-centered and having a bar for evidence and an expectation that you’re working toward evidence-based practice I think is a really difficult thing to learn to navigate.

There needs to be structure and processes while also allowing for site-based autonomy and alignment to district priorities, expectations, and ways of working. If the improvement science ethos takes hold in the system, the school-based decision-making will be easier to organize. Jack pointed out the following:

I think you need ways of helping people feel empowered and ownership to improve things, to fix things, to meet that standard better, and to try, and that system sort of rewards that. To do that, you need a disciplined culture… And that is not up to any individual. It’s what we are collectively trying to do and a sense of mission and purpose to do so.

Importantly, findings from interview data in this section highlighted that organizing improvement work includes considerations well beyond organizing tasks. When district and school leaders organize the work, attention to how the routines operate is critical. Participants revealed that to dedicate the appropriate time, attention, and personnel to improvement, the work must fit into existing structures as well as consider how the new responsibilities associated with improvement will likely need to result in other responsibilities being shifted or jettisoned.

In addition, participants mentioned that leadership mindsets are also a component of organizing the work. Leaders can impact organizational structures and norms by leading by example (e.g., engaged in the improvement process themselves and learning from failure). When planning to organize and implement the work, attention to leadership and structures and responsibilities need appropriate attention to set the improvement leaders and team(s) up for success when moving into implementation of the improvement work.

3.5. Implementing the Work

We found four implementation practices that support organizational growth. These practices support the school-based practitioners in taking up the learning necessary for implementing improvement science. These help to streamline learning and growth.

3.5.1. Practice 1: Learn It, Try It, Learn More

Participants revealed that the successful implementation of improvement science begins with the first major aspect of the work: learning the process while solving an authentic problem. The use of improvement science begins with gaining initial knowledge of the improvement process by understanding the problem, developing a theory of improvement, designing, and conducting iterative tests of a change idea. At the same time, the organization seeks to solve an actual problem troubling their setting. Some of the experts we interviewed pointed out that it is important to resist the urge to focus on training the improvers as the goal of the work. Instead, one must stay focused on improvement and outcomes to learn the process.

This becomes a learning-by-doing process, which is a potentially efficient approach as it simultaneously addresses a problem of practice and helps those engaged in addressing it become more effective improvers. After establishing some grounding knowledge of the process and core principles of improvement, it is best to engage in the process and learn more through this participation. One interviewee, Jamie, a NIC facilitator, suggested the following: “I will say that I deeply believe the only way to learn improvement science is to improve things over and over. Like you have to actually improve things. You can’t go learn about it. You have to go do it”. The people from within the organization will learn about improvement science processes by improving something and then use those improvement outcomes to learn more about both the process and the problem. She went on to add, “Don’t worry about doing it right. Just do it. You will learn from doing it. Just try stuff. It’s okay… If you’re learning, you’re doing it right. It’s learning”. She emphasized learning as an outcome of improvement science. Similarly, as Fullan and colleagues noted, learning during implementation is a mechanism to support effective change [50].

Interviewees noted the delicate balance between learning to do improvement science and solving an authentic problem (particularly for schools) in organizations is a highly complex task. The process of learning in organizations and leading systemic changes are complex tasks [2]. When leading change is combined with learning a new process, the level of complexity increases. For that reason, experienced improvement scientists believe the best way for organizations to engage in improvement science is to learn by improving and then use those results to learn more.

This interweaving and cyclical process of simultaneous learning and improving necessitates a feedback loop to develop examples of improvement to demonstrate and help spread implementation. Sarah, an improvement science expert on data, pointed out the following:

I think what people really latch onto is the experience of learning. It’s like the aha moments. I think getting the work up and going, if you could sort of build on those, like those aha moments to happen for folks, that’s a very addictive thing.

A ‘learn-do-learn more’ approach can be supported by sharing quick wins and bright spots in the work. By celebrating areas of success, people will become more curious about the process and deepen their understanding of the work. As Ericka, an NIC facilitator and improvement coach, explained, “I think to sustain improvement activity, people have to keep seeing that they’re learning, and others are learning and sharing”. Sharing learning is key. One way to spread learning throughout a complex organization like a school district is to develop in-house examples of improvement to help spread implementation. Suzy, an improvement science expert with NIC experience, explained how this requires data analysis:

That actually allows you to identify some bright spots, and really going and studying what the heck they’re doing. I think people do bright spotting, but they don’t get down to the concrete minutiae… Let’s capitalize on what we know about what they’re doing and then get that to spread because people will also be more likely to pick up stuff that’s in-house… then I think you can also take a handful of promising practices and get those things to spread while you’re then also continuing to do other work that you need to do to understand the system.

The learning process will bolster the work, encourage participation, create collective beliefs, and ultimately allow for promising practices to spread through the system.

3.5.2. Practice 2: Focus on the ‘Why’ as Opposed to Only the ‘What’

As part of the improvement science process, improvers should make explicit the improvement aim or outcome of the work and attach all the processes, practices, and mindsets to that aim. The participants in this study suggested it is important to have a north star to guide implementation. Jacob, who has worked on improvement projects of small and large scale for districts and schools, explained:

That aim itself is going to dictate what every other thing is, and we can’t start building any of the stuff or thinking through any of the things without knowing what we’re trying to accomplish. So, I would say that’s central to everything and then it stays consistent over time… it’s not something that’s fly-by-night; we’re going to continue to chip away at this and have this north star that we are continuing to work at.

With the North Star, or the ‘why’ established, improvers will want to stay the course and maintain focus on results (or the ‘what’) always reflecting on the reason for the work as well as the changes in the organization. Jamie, who runs NICs across districts, explained, “I do think you have to be explicit about when we say, ‘We’re going to redesign systems’ we’re going to redesign systems with Black and Brown students at the center to ensure that the system works for them”. Amanda also added, “So interrupting inequities, I would say is the reason, the why behind why we do the work. I have a lot of hope and optimism that improvement science can do that, can help redesign systems”. One way to ensure the focus on ‘why’ is maintained is to use the model for improvement (i.e., what are we trying to accomplish, how will we know that change is an improvement, and what changes can we make that are an improvement) and infuse empathy and equity deeply into that model [6]. Empathy is ensured by empathy interviews or short interviews that reveal individuals’ experiences with the problem at hand. Applying an equity orientation to improvement is a way to center the why. This emphasis on equity needs to be carried out purposefully. Amanda went on to share the point:

I think improvement science wasn’t originally designed to address inequities. it’s very easy to take an improvement science approach and to miss the equity completely. It’s not built into improvement science methods. I think people who are trying to use improvement science and equity right now are figuring out what that integration looks like.

The ‘why’ is not inherent to the tools and process but needs to be at the forefront of the work; the researchers believe that improvement that does not advance equity is doing little to change routines. Amanda suggested using empathy data, process and systems mapping, small-scale tests of change (e.g., PDSA), and data practices to bolster an equity lens and improvement lens. Penelope, who was a district leader who also works for an external organization, suggested:

I never talk about initiatives. Improvement is the work that we’re doing, and we will never be done. Improvement has a start, but it doesn’t have a finish. And we are going to continue to dig in until we’re answering the questions of what we value most for everyone we serve.

Grounding the implementation in shared core values gives the improvers the opportunity to engage in collective action. Penelope also said the following:

When you’ve got everyone pulling on the same end of the rope, and it doesn’t matter what you say your values are, but when you’ve identified your core values as a team and you’re consciously celebrating those, that feeds your energy for that long haul at work.

A ‘why’ to guide the work helps to keep everyone working collectively in pursuit of shared goals.

3.5.3. Practice 3: Keep it Simple

This complex work requires seeing the system or conditions that hold the problem in place, shifting mental models to ensure change, and creating enabling conditions centered on the use of improvement science, including building trust to work together in ways that might seem profoundly different than previous approaches to problems. To ensure that implementation can be successful, participants shared that the process needs to be communicated in practical terms and framed for classroom teachers, school leaders, and their improvement team members. Barry, a leader in K-12 improvement science, recommended to “de-emphasize improvement lingo, language, and tools and still try to maintain mindsets”. To achieve this, improvement leaders must avoid being too technical or theoretical. The more the process aligns with existing district goals, language, and ways of knowing, the easier it will be to adopt an improvement science lens. If there is a common language within an improvement team or NIC, the improvement language can be adapted to match the existing language. Jacob talked about a school that “doesn’t use the language of improvement science at all” and could even be “using improvement methodology without even realizing it”. However, this organization was able to take a systems lens, build data infrastructure, and own the problem, resulting in improved outcomes. He noted that this “shows how much you can boil this down”.

Similarly, participants concurred that improvers should not be over-reliant on tools. The tools are an important part of the work, but they are used in service of a larger goal or purpose. When possible, they suggested removing barriers (like PDSA tracking forms), so the improvers can see the process work without getting in the weeds.

3.5.4. Practice 4: Differentiate Learning by Role

An important aspect of this work is gaining the knowledge necessary to sustain the use of improvement science in an organization as well as understanding individuals’ roles within the work. Participants suggested that knowledge should be differentiated throughout the organization and should meet people where they are developmentally while centering the responsibilities associated with their role. Several participants suggested it is important to use a book like Learning to Improve [2] or Improvement Science: A Primer [6] to inform those leading the work, but it might not be necessary for all members of the organization to engage in that learning if a few key people understand the model of improvement and core principles of improvement science. Stacie, a NIC facilitator and district leader, said, “It’s about doing. I think there’s an introductory level of training that people need to know, and I think what people need to know is different”. The identified problem for improvement should guide what people need to know and do. As Suzy suggested, “Pick something and stay focused on it for an extended period of time. And then think about what it is that all the different people in the different levels of the system need to know or understand in order to solve the problem collectively”. The level and type of knowledge needed are based on the role in the system. Different people need to have different levels of understanding. Amanda, an experienced improvement coach, explained:

It’s fundamentally about teams, frontline teams. So, you’ve got the teams, and they need to know some things. That team often has a lead who might know just a little bit more than the rest of the team. And you’ve got coaches. Coaches support the teams, and their job is to build the learning capacity of the teams… If you’re an improvement specialist, how much you need to know about the theory of improvement shifts quite a bit depending on where you sit. The frontline doesn’t need to know tons about theory. There’re some key mindset shifts, and this is how we’re going to look differently than we’ve done before. But if you’re going to be a coach and especially if you’re going to be an employment specialist, you need to know the deep, underlying ideas that are going beyond, because it’s never linear.

District- and building-level leaders require basic knowledge of the process and need to support the development of structures and routines as well as the use of resources (e.g., time and money). Suzy noted, “at the district level, you’ve got different people that are head of different departments that need to then be able to see the through-line”. These district leaders need to understand cross-cutting themes and elements and how those structures and routines can hinder or enable the improvement science process. Participants suggested that hub members and team coaches may want external training (or a coach) to help gain the level of knowledge necessary to learn the work. They suggested that improvement coaches needed to have deep knowledge and experience in what they are coaching. Coaches support the teams, and their job is to build the learning capacity of the teams.

When leading improvement science, the coaches and hub team members will want to determine what knowledge is new and focus coaching on those areas. Amanda expressed that this not only includes improvement but also includes mindsets:

Personally, what’s important is that they can think about their own thinking and their own biases, their own identities, their own emotions. So, you’ve got a little bit of that… The biggest thing I usually look for on the frontline team is the ability to have conversations, like really good candid and curious conversations, even about hard topics like racial inequities… the empathy interview is pretty important, process mapping is pretty important… PDSA and regular use of data are going to be helpful.

The improvement routines, critical self-reflection, and leading and engaging in conversation about inequities were examples of new knowledge that this seasoned improvement coach has found needs to be developed.

Improvement science is carried out in teams and those teams can include teachers, staff, and school leaders. Participants shared that it is important to encourage informal leadership roles while remembering that leadership is not an inherent skill, and it needs to be encouraged and developed, especially with teacher leaders. That team should have a leader who might know a little bit more than the rest of the team. Those leaders will need support to lead effectively. As Suzy suggested, principals and other school-level leaders must focus on the following question: “What are all the different processes that are there to support kids and to support teachers into being able to improve their instruction?”

3.6. Sustaining the Work

The interview data revealed three key areas for planning how to spread improvement through a system: a coaching plan, a growth and succession plan, and a knowledge management plan. These plans undergird the implementation of improvement science in a district or network and support the sustainability of the improvement work. These plans are often role-dependent.

3.6.1. Coaching Plan

In many cases, schools are supported in their learning by improvement coaches who support learning the process, developing and testing change ideas, and establishing routines for the work. To be successful with an organizational-wide approach to improvement science, participants shared that it is wise to embed improvement coaches throughout the system, including within divisions or units throughout the district. People at each level of the system would benefit from having a coach support their roles and responsibilities in a systems-wide approach to improvement. Amanda suggested the following:

I think the piece that is really missing is the coaches… And learning how to coach improvement is really hard. And you need to actually invest in people who can coach that. And it’s a huge resource for your organization, but I feel like people think you can go to a three-day workshop, and you can know how to do it from them.

The coaches guide the work. Penelope noted:

Usually, the coach spends a decent amount of time on the launch of the team… you’ve built the conversational capacity of the team, the culture of the team, some of the information skills… Then you sort of coach along the way. Helping to design the improvement journey is pretty important.

Penelope’s comment focused on how coaches are not only key to sustaining improvement but that during the organization and implementation stages of the work, coaches can be a critical resource to shepherd the improvement efforts along the way. Her comment highlights that coaches are important stakeholders to involve even after the implementation stage to sustain the work. Coaching is key to implementing and sustaining improvement science in organizations. Planning for that coaching will further support sustainability.

3.6.2. Growth Management and Succession Plan

One of the biggest hindrances to institutionalizing improvement science is the turnover or churn that takes place in schools and districts. If the system is developed to be too dependent on individuals and vulnerable to people shifting in and out of roles, it will be difficult to sustain the work [51]. Jacob and Penelope talked about the deep bench development. Jacob stated, “if we do have turnover, we don’t have to start from scratch; we just keep moving and everybody understands what we’re doing”. Penelope explained that succession planning is not simply responding to open positions:

Leadership at every level. You’re developing leaders, so they develop their leaders… Succession planning is really about developing your deep bench, not about realizing somebody’s going to leave you in six months and you’ve got to prepare to fill that job… we’re training people to be skilled and have that mindset and that skillset. The mindset is sitting up here and it has to live with everyone.

By building improvement science into the district organizational routines, leaders can avoid halting improvement work when turnover inevitably happens.

To sustain improvement processes system-wide despite this churn, participants suggested a growth management and succession plan. These plans should include multi-level considerations from the school level up to the district level and the district level down to the school level. As previously mentioned, the coordination of overarching values and articulation of the ‘why’ behind an improvement is important. A common narrative can bring cohesion to the plan details. In addition, participants suggested that the leaders in the organization need to determine who needs to know what and when (in service of the aim) and then determine what is needed and how elements are needed to make the execution of the plan realistic. They mention the need to ensure informal leaders, or those people not in official leadership roles at the moment who will make up future leaders, are informed and aware of the work. Penelope advised that:

It’s got to be based on core principles and it’s got to be ongoing, and you’ve got to get hardwired. So, you can have as many improvement teams going, but if it doesn’t affect that backbone, building that backbone, if it doesn’t align to how they onboard and they give feedback, at a system level, it’ll be temporary. If it doesn’t align to how they’re developing their informal leaders, it will evaporate when the leader leaves.

As part of the growth management plan, routines need to be considered that help to encourage the institutionalization of improvement science across a district. Denny, a top researcher on organizations and improvement science, mentioned:

Sometimes the context itself is just not ready for it. What I mean is, there’s no standing meeting schedules, there’s no norms around getting together in preparing for meetings, the instructional programs are all over the place and nobody’s doing anything roughly similar.

These aspects need to be part of a plan for growing and expanding to ensure that the work can be sustained (i.e., institutionalized).

While the organizational systems and levels will engage in growth, building individuals’ growth to move the system forward will also need consideration in the plan. In other words, participants affirmed that a growth management plan needs to include collective growth, but it should also include a focus on individual growth. Denny explained:

There are people in an improvement enterprise that are at all different years… this really complicates the leadership function because it means that there isn’t really a developmental sequence. There are multiple developments… It can be a kind of dominant progression that things move along that takes firmer form as the enterprise becomes more socially stable but on route to some sort of base-level and social stability, you’re managing multiple individual developmental trajectories.

The complexity of this improvement work requires thinking about both individual trajectories for gaining skills, mindsets, and practices for improvement and considering how individuals at various levels of the system leverage those skills, mindsets, and practices collectively. Connected to the growth management plan, participants suggested the need for a succession plan with a deep bench of individuals who are familiar with the improvement process and trained at the appropriate level for their role.

3.6.3. Knowledge Management Plan

To sustain the work, districts need a knowledge management plan, which should focus on knowledge management between the school, the hub, and/or the district. The plans go beyond the measurement structure in the NIC and are about diffusing knowledge districtwide. Stacie explained that knowledge management of improvement in the system

is really critical. It’s not just all going to organically, magically happen, there’s that visual of like the PDSA cycles rolling up the ramp. It just looks like, ‘Oh, you keep doing PDSA cycles, this is going to roll up this ramp.’ And it doesn’t, it takes really concerted effort to gather the knowledge that is being developed on the ground level, identifying what’s sticking and what’s not sticking, consolidating that, putting it back out there. That’s a big part of this work.

Multiple participants shared their stories of the evolving nature of this work and how the need for knowledge management was revealed from missed opportunities to learn from their improvement work. These missed opportunities were due to not having a strong data infrastructure or plan for distributing the learning. For example, participants from one organization known for its work on knowledge management told of how they excelled at innovation but lacked disciplined inquiry, leading to disparate improvement projects and truncated learning. Two participants discussed the following in a group interview:

Sarah: I do think measurement matters a lot and it’s oftentimes the thing that brings people around the table; this acknowledgement of here’s where we are and here’s where we want to be and there’s this gap and we’re trying to close this gap. I think, if you get those sort of measurement systems up in place and people tracking the information {about} things that matter, and on an ongoing basis, that infrastructure can survive people coming and going in ways that when we’re not there, or if it’s entirely sort of individual-dependent it’s harder. So, I think there’s a role that measurement can play and sort of keeping that focus constant.

Jack: What I’d say is different, but very related, which is, I think, establishing the standard work of leadership. Getting really clear about what the standard routines are that leaders do and a big part of that routine should be making sense of that data. And then seeing things in that data that lead leaders to go and spend time in or on the ‘front line’ of wherever it is that they’re going, trying to see and understand where the work is happening, where the value is happening and, ultimately, all the value or almost all the value in school district is happening in classrooms with kids. And so, the more removed you are from that the less you are going to have a window into what needs to be happening so figuring out how to do that and then hopefully, through that building [knowledge]. There’s so much waste that just happens in the system, so much rework, so many things that are happening that aren’t adding value.

To sustain organizational change, participants felt there must be a plan to leverage data and learning into knowledge about individual change ideas and the context-specific enabling conditions for continuous improvement. The participants suggest these data practices need to become organizational routines that determine what is measured and what can be learned from those measures, what knowledge needs to be shared, and how leaders will use that knowledge for future decision-making.

4. Discussion

This study seeks to support lasting, institutionalized change by adding to our understanding of how a district or educational organization can prepare for the organizational institutionalization of improvement science. We have largely discussed improvement science in the context of NICs, but it can be carried out at individual schools or at the district level as well. The Framework for Organizing, Implementing, and Sustaining Institutionalized Improvement Science addresses aspects of leadership, organizational change, and enabling structures that allow improvement science to take hold in different formats. Without the structure and routine for ensuring that continuous improvement is supported in a district, the use of improvement science and the use of NICs run the risk of becoming just another initiative for school improvement or reform that does not take hold in the system and is impossible to sustain [2].

The leadership, organizational change factors, and enabling structures identified as a challenge for the implementation of improvement science include time, resources, skill, and will. This framework responds to the previously described implementation challenges in the following ways:

• The will to implement despite challenges with time, resources, and initial hesitation [13,14]:

  o

  Centered on the schools (organizing);

  o

  A mix of top-down and bottom-up learning (organizing);

  o

  Integrated into existing work (organizing);

  o

  Focus on the why (implementing);

  o

  Differentiate learning by role (implementing).

• The skill necessary for this work is differentiated and is gained by learning by doing [14]:

  o

  Learn it, try it, learn more (implementing);

  o

  Keep it simple (implementing).

• Time for shifting ways of thinking, being, and doing [12]:

  o

  Focused on the why (implementing);

  o

  Learn it, try it, learn more (implementing);

  o

  Keep it simple (implementing).

• New culture, capacities, and resources to allow for improvement [16]:

  o

  Integrated into existing work (organizing);

  o

  Keep it simple (implementing);

  o

  Coaching plan (sustaining);

  o

  Knowledge management plan (sustaining);

  o

  Growth and succession plan (sustaining).

4.1. Collaborative Structures

We see opportunities for this framework to be used in conjunction with other improvement frameworks such as Russell and colleagues’ framework, which is specifically about launching NICs, which are a prominent way to conduct improvement science within districts [11]. This study reinforced aspects of Russell and colleagues’ framework that need to be developed through parallel efforts between the NIC (e.g., analytic structure) and throughout the district or educational organization where the NIC is operating (e.g., knowledge management). Our framework also applies to other implementation models that involve using improvement science outside of the NIC structure; some districts and educational organizations are using improvement science in conjunction with school improvement planning and may encourage system-wide learning but have schools define their problem of practice. For instance, schools may use an improvement science process by learning the methods and creating their own theory of practice improvement (at the school level) to address their own problems of practice (no common aim) and measure that progress at the school level (no common analytical structure). However, the district needs to be prepared to support the individual efforts across these inter-organizational contexts by dedicating resources and supporting the prioritizing of their improvement efforts, for example.

4.2. Supporting Multi-Level Organizational Change in School Districts

Throughout the data collection and analysis and subsequent framework development, holding a systems thinking lens was evident. Improvement experts emphasized how important it was to examine the routines and processes of the system to be able to make changes for improvement. Further, the evidence mirrors literature centered on district-level actors operating as a learning organization [18,27]. Understanding the systems involved in the organization can be leveraged in the planning, organizing, and sustaining of the improvement work because the NIC and hub members will be positioned to see the “interrelated parts that work together in some fashion and impact each other” [24] (p. 806), and will therefore be able to design and incorporate processes and norms to work effectively within the system to organize and advance the improvement work.

Participants in this study made it clear that improvement work is complex. This study revealed there are numerous intentional considerations to attend to at the different junctures of this critical and complex work to organize, implement, and sustain improvement science processes. It involves many people, across multiple levels, and changes in organizational routines and structures. Supporting multi-level organizational changes is challenging (e.g., [53,54,55]). Intentionality in how to institutionalize change processes for improvement can help support lasting, systematic changes within an organization. Building the capacity for improvement by developing a ‘deep bench’ ensures that many are poised to carry out and lead the work and helps sustain the work even if someone involved leaves their professional position. As Welsh and colleagues found in their research on schools as learning organizations, “Organizations that have weaker infrastructures often fall prey to leadership changes and are not able to sustain the cultural shift” [35] (p. 375). While the present framework design is responsive to a large K-12 school district, we predict districts or organizations of various sizes and in various contexts could find utility in attuning to the components of supporting improvers in planning, implementing, and sustaining their own improvement work.

4.3. Implications for Practice, Policy, and Research

This study holds direct implications for practice, particularly because this project was born out of the district’s goal to create a NIC to make their systems work more effectively for marginalized populations of students. The framework is currently being applied directly to this goal. Those who are charged to lead improvement are called to be purposeful in guiding improvers in identifying a problem of practice in their context that is threatening the equitable educational opportunity for students, particularly those groups of students who are consistently marginalized (e.g., students of color, ELs). The participants identified that providing improvers with the necessary space, security, and opportunities to meaningfully engage in improving an authentic problem of practice are some of the necessary organizational ingredients to plan for, implement, and sustain the work [12].

Another area with implications for practice is the balance between new and existing structures and processes. Some may be concerned that the suggestion to build from existing priorities, expectations, and ways of working could limit new ways of being and thinking. It is important that those in charge of improvement science, whether within or outside of the district or in partnership, stay true to implementing improvement science with integrity while also managing the amount of new culture, skills, capabilities, and conditions that educators must learn. By meeting people where they are, focusing on the ‘why’, and keeping schools and students at the center, there is a greater chance of institutionalizing the work [13,16].

Closely connected to practice is how policy can support this work. Honig noted that policy can be leveraged to develop a mindset and organizational routines for district-level leaders to move beyond the managerial responsibilities to meaningfully engage in supporting schools in improving systems to improve student learning [18]. Policymakers—particularly those serving at the local level—should consider ways to support the organization of, and create the enabling conditions for, improvement work.

For instance, the prominence of the need for coaching is an area for further policy and practice exploration. Coaching as an investment connects to the resource-intensive requirement of improvement to take hold [11]. Most organizations supporting districts with improvement science use a coaching model and have found this approach to be an effective way to embed professional learning into the system, thus focusing on lasting change instead of shiny new initiatives [50]. Much of this work has been supported by grant funds; however, in our experience, these do not need to be large grants. There needs to be enough financial support to commit to improvement coaching, and this work does need to be led by someone with enough knowledge of improvement science to guide the front-line workers. The use of coaches to launch the work could potentially evolve into improvement coaching embedded as a leadership activity or in the work of district-level leaders and managers who support schools leveraging a ‘train the trainer’ model.

Finally, the current study offers a foundation for further exploration of the relationships between organizing, implementing, and sustaining improvement work. The results call for future research to examine the experiences of improvers operating at different levels of the system. For instance, a mixed methods study that seeks evidence of progress on an improvement team’s aim while also capturing the experiences of those leading, engaging with, and experiencing the improvement approach would illuminate areas where the proposed framework needs expansion or contraction. Future studies that follow an organization in its planning for and implementation of improvement science over time to capture the opportunities and challenges present as the organization scales and sustains its work would help both researchers and educational professionals better understand the organizational considerations (e.g., structures, leadership) that emerge, including those that could impact the planning and implementing portions of the current framework. We also see the potential to expand this framework into a field guide that explicates the ‘how to’ of each element of this framework for district leaders.

4.4. Limitations

This study is not without limitations. The first limitation is the lack of diversity in the participant sample. Many improvement science experts received their start at the same organization, and most were white. This indeed limited the breadth of perspectives used to develop this framework. This framework may need further development to understand how non-white educators might experience the requirements, assumptions, language, and outcomes of the framework. Future research should move beyond the organizations tapped to seek voices from a variety of backgrounds and professional contexts. A second limitation is that this framework was developed to support one specific district under a research grant, but the refinement of this framework would be strengthened by future research to examine how it aligns or diverges from the organizational and system considerations in different contexts (e.g., small, rural schools, and non-education organizations). Finally, some of our data lacked specificity and concrete examples. Further research could capture the ‘how’ of this work.

5. Conclusions

This study contributes to our understanding of the implementation of improvement science in schools and districts to ensure sustainable institutionalization. Existing research on improvement science highlights the importance of having organizational routines for the work [2], establishing collaborative structures to learn during the work (e.g., NICs) [11,45], providing guidance on how to organize the work [11], and inviting fundamental changes to educational systems to meet the needs of those who are traditionally marginalized [6,11,23]. This framework expands upon the existing literature to identify and organize the type of work considerations that need attention at different stages of the incorporation of improvement science within an organization.