Using Action Research to Develop a Professional Development and Coaching Package for Educators within Inclusive Reading Instruction

Abstract

One foundational aspect within the essential skill of literacy is reading comprehension. While students can learn comprehension strategies in a variety of settings, students with disabilities have shown increased success in inclusive classrooms with instruction supported by both general education and special education staff. To address the needs to increase the reading comprehension outcomes for students with intellectual disability as well as increase the fidelity of use of evidence-based practices by school staff (general education teachers, special education teachers and paraprofessionals) within instruction, an interactive action research study was implemented in five schools in one school district in the United States. The study was designed to use action research to create a model of professional development with extensive coaching support surrounding evidence-based practices as well as the development of resources to support co-planning and adaptations within instruction in inclusive classrooms. Results indicated that both purposes were met within the study with several implications for practice as well as the development of a second iteration of the model using student outcomes and frequent educator feedback.

1. Introduction

The development of literacy skills is essential to ensuring high quality of life outcomes for individuals with intellectual disability (ID), including increased post school employment, access to leisure activities, and independence in daily living [1]. In the United States, the Individuals with Disabilities Act [2] is a federal law which requires that all students with disabilities have access to the general curriculum content in the least restrictive environment (i.e., with their nondisabled peers to the maximum extent appropriate). Although a version of this law requiring access has been in place since 1997, there continues to be wide variation by state in the methods of delivery and quality of academic instruction provided in general education settings. There are numerous potential reasons as to why this variability exists (e.g., teacher training, expectations, lack of resources) within models of instruction.

In a foundational piece on literacy instruction for students with ID, Browder, Ahlgrim-Delzell et al. [3] presented a conceptual model for teaching literacy to students with ID that addressed increased independence as a reader and increased access to literature. The authors model for increased access to literature includes: (a) opportunities to access literature, including adapted texts, time for literacy, and read alouds either by people or technology; and (b) instruction to access the text, including text awareness, task analysis, vocabulary instruction, and listening comprehension [3]. This model has been studied extensively to teach comprehension of both narrative and informational text across academic content areas, e.g., [4,5,6,7,8,9,10] and includes a treatment package of evidence-based practices (EBP) for academic instruction for students with ID including adapted texts made accessible through read alouds (also called story-based lessons or shared stories [11], explicit vocabulary instruction using constant time delay [CTD; 6], task analytic instruction with systematic prompting—specifically a modified system of least prompts (SLP) and feedback [12,13], and graphic organizers [14].

Despite recent advances in literacy instruction for students with ID, data suggest the majority of these students receive academic instruction in segregated settings [15,16]. For example, the National Center for Educational Statistics [17] reported that 19% of students with ID in 2020 were included in the general education classroom 80 percent or more of the day while 47% of these students were include 40 percent or less of the day. Chadwick et al. [18] outlined the impact of COVID-19 which further reduced inclusive opportunities for individuals with ID. Although there has been some progress in increasing the number of students with ID educated in the general education classroom over the past decade, these improvements are not commensurate with those of other disability groups [19,20]. This is troublesome for students with ID, particularly in the area of literacy as it may prevent meaningful access to the general curriculum [21]. Ruppar [22] found that most of students’ literacy experiences in self-contained classes were an “ad-hoc set of activities” and did not reflect a cohesive curriculum with a clear scope and sequence. More recently, the author and her colleagues found that students were 10 times more likely to be exposed to academic literacy when peers without disabilities were present [23].

1.1. Reading Comprehension and Inclusion

Reading comprehension is one of the most essential life skills for all students. The comprehension of written text has been referred to as the “essence” of reading [24], and is essential to reaping the full benefit of interacting with literature. Beyond academics, comprehension is one of the most critical skills for independent functioning as adults [25,26]. Fortunately, several research teams have demonstrated that weaknesses in reading comprehension by students with ID are amenable to intervention and have established evidence for an emerging set of practices including read alouds [7], embedded instruction [4,27] peer supports [7,25,27,28,29] and graphic organizers [10,29]. Additionally, specific instructions have been developed for adapting grade-level texts to an individual’s listening comprehension level, such as shortening and summarizing the text, using a predictable structure, augmenting the text with visual supports and physical supports if needed, and providing response options [30].

The delivery of literacy instruction including reading comprehension in inclusive environments may provide additional benefits to students with ID. Researchers have demonstrated a relationship between time spent in inclusive environments and school and post-school outcomes. Additionally, they have shown that inclusive opportunities can lead to increased expectations for learning by students with ID [19,31]. Data also indicate that students with ID learn more content, at faster acquisition rates, and have high levels of active participation in inclusive learning environments when instruction is embedded during naturally occurring contexts and activities [31]. The available literature suggests that increased opportunities for literacy instruction in inclusive contexts may be related to enhanced quality of life outcomes; thus, there is a need to establish models for facilitating a shift in practice towards the effective delivery of literacy instruction for students with ID in inclusive settings.

1.2. Evidence-Based Practice

One barrier to the inclusion of more students with ID is the lack of training of all educational staff in the use of EBPs during instruction. In the last decade, EBPs for use within instruction of students with ID in the United States continue to be identified [32,33,34]. Unfortunately, the identification of EBPs by researchers does not ensure that these practices are widely adopted and accurately implemented. Researchers have demonstrated that high quality training may not be sufficient for producing sustained teacher behavior change [35]. Additionally, general education teachers and special education paraprofessionals are typically not privy to professional development (PD) related to these instructional practices.

Behavioral skills training (BST) is one way to promote fidelity of implementation [36] and has been demonstrated effective in training teachers to implement [37]. BST comprises four key components (a) the provision of written directions, (b) modeling, (c) rehearsal, and (d) feedback, components consistent with coaching literature in special education [38,39]. Further, learners are required to practice until they meet a predetermined criterion before they implement procedures with students.

1.3. Using Action Research to Build a Model

Action research plays a vital role in education. Action research, through its cycles of planning, observing, and reflecting collaboratively by stakeholders, seeks to improve social issues including the identification of effective educational practices [40,41,42]. Hine [43] explained that, within the field of education, action research can be used to improve both the experiences of students and the professional development of teachers including benefits such as the participatory nature of educators and students (i.e., empowerment) in a systematic reflective process. Action research supports the closing the research to practice gap, a gap which often interrupts the use of effective practice in schools with identifiable needs [43].

In this project, literacy instruction, including reading comprehension, for students with ID has been a priority for a local, large, urban school system in the southeastern United States with a long-standing partnership with university-based researchers. Working together we identified the aim of the project—to create greater access to literacy content with a specific focus on reading comprehension using EBP with fidelity during instruction by educators in both segregated and inclusive classrooms using a sustainable coaching program. A classroom-based, practical, participatory action research design [42] best suited the needs of this project to solve this local problem. In this partnership, university researchers contributed their knowledge of EBPs along with EBP training and coaching while classroom educators contributed the practical implementation-related aspects to the project to develop a model of EBP training and coaching that was useful, meaningful, and feasible for educators to implement EBPs during classroom instruction and improve student outcomes.

1.4. Aims of the Paper and Research Questions

The action research project was designed to identify how to best support district personnel in classrooms that served students with ID to implement EBPs with fidelity during instruction targeting reading comprehension. Specifically, the project was constructed to determine the impact of a series of PD and weekly subsequent coaching sessions on the instructional performance of the special education classroom staff (teachers and paraprofessionals) and academic performance students with ID. Specifically, the research questions were:

• What is the impact of EBP PD and coaching on educators’ ability to implement EBP during reading comprehension instruction?
• What is the impact of the educator implementation of EBPs in special education settings on student reading comprehension outcomes?
• What is the impact of educator implementation of EBPs in inclusive settings on student reading comprehension outcomes?
• What is the usability and acceptability of EBP PD and coaching by educators?

2. Materials and Methods

This action research project used a convergent mixed methods design where qualitative and quantitative data were collected and analyzed at the same time. Quantitative data were used to inform the qualitative focus group questions. The qualitative data were used to inform the subsequent quantitative, closed-ended survey questions. Both types of data were used to inform the action research process to improve the impact and acceptability of the coaching packages.

2.1. Participants

This project was conducted in 5 schools (3 elementary and 2 middle) from a large, urban public school system in the southeastern United States. Primary participants consisted of 6 special education teachers, 6 paraprofessionals, and 6 general education teachers and 37 students. One special education teacher, one general education teacher, and one paraprofessional formed a triad.

Table 1. Teacher and paraeducator characteristics.

Characteristics	Elementary		Middle
	N	%	N	%
Gender
Female	12	100	6	100
Ethnicity
African American	2	16.7	2	33.3
Hispanic	2	16.7	4	66.7
White	8	66.7
Job Title
General Education Teacher	4	33.3	2	33.3
Special Education Teacher	4	33.3	2	33.3
Paraprofessional	4	33.3	2	33.3
Teaching Experience
1–5 years	4	33.3	1	16.7
6–10 years	3	25.0	2	33.3
11–15 years	3	25.0	1	16.7
16–20 years	1	0.08	2	33.3
Missing	1	0.08	-	-
Highest Degree
High School	1	8.3	0	0
Bachelor’s degree	7	58.3	4	66.7
Master’s Degree	4	33.3	2	33.3

displays the educator characteristics. Most of the teachers and paraprofessionals were White females with a bachelor’s degree, ranging from 1 to 20 years of teaching experience.

All students received instruction in a separate classroom designed for students with ID.

Table 2. Student characteristics.

Characteristics	Elementary		Middle
	N	%	N	%
Gender
Female	2	9.1	7	43.8
Male	20	90.9	9	56.3
Ethnicity
White	5	22.7	7	46.7
African American	6	27.3	5	33.3
Hispanic	9	40.9	2	13.3
Others	2	9.1	1	6.7
English Language Learner
Yes	3	13.6	1	6.7
Estimated IQ
80–71	4	18.2	2	12.5
70–55	9	40.9	3	20.0
54–40	4	18.2	8	53.3
Unable to estimate IQ	2	9.1	1	6.7
Parent did not consent to disclose	3	13.6	1	6.7
Augmentative Communication System
yes	1	4.5	4	26.7
Lesson Engagement
Does not engage in lessons	2	9.1	1	6.7
Engages in lesson 1–10 min	8	36.4	2	13.3
Engages in lesson 11–20 min	6	27.3	5	33.3
Engages in lesson 20+ min	6	27.3	7	43.8
Engagement Support
Resists engagement	1	4.5	0	0.0
Engages only when prompted	9	40.9	9	60.0
Initiates and sustains interactions in lessons	12	54.5	6	40.0
Expressive Communication
Verbal expression	21	95.5	13	86.7
Nonverbal expression	1	4.5	2	13.3
Receptive Language
Alerts to sensory input, requires physical  assistance to follow simple directions	0	0.0	1	6.7
Requires additional cues to follow 1–2 step  directions	9	36.4	7	43.8
Independently follows 1–2 step directions	13	59.1	7	43.8
Reading
No observable awareness of words in print	0	0.0	2	13.3
Aware of text, follows directionality,  letter distinctions, or echoes reader	4	18.2	3	20.0
Reads basic sight words	8	36.4	2	13.3
Reads simple sentences, directions, lists	6	27.3	1	6.7
Reads fluently	4	18.2	7	43.8

displays the student characteristics for whom parental consent was obtained. Most of the students were male (75.7%) with diverse ethnic backgrounds (White 32.4%, African American 29.7% or Hispanic 27%) with some verbal communication skills (91.9%) and basic reading skills (76.4%). Most students had an estimated IQ ranging from 70–55 (29.7%) or 54–40 (27.0%) as determined by psychological test results. IQ is estimated because students were noted to have difficulty in participating in the rigid administration procedures of IQ tests and lacked the verbal fluency needed to respond. Secondary participants consisted of 4 district-level staff of the school system and 9 experts in teaching reading to students with ID. No data were collected from the secondary participants.

2.2. Action Research Process

The project consisted of 2 phases consisting of training and coaching. The goal of Phase 1 was to train special education teachers and their paraprofessionals on the use of EBPs for reading comprehension instruction. Phase 1 began with the secondary participants collaborating with the university team to create the EBP reading comprehension training workshop. The university team drafted the content of the training workshop. The secondary participants reviewed and made recommendations that were incorporated into the final training package. The EBP training consisted of a 2-day workshop. The first day of training targeted EBPs within explicit and systematic instruction (e.g., constant time delay [CTD] for vocabulary instruction and the system of least prompts [SLP] for comprehension). CTD is a systematic and errorless learning strategy for a discrete skill in which stimulus control is transferred using a delayed prompt [44]. The SLP includes a hierarchy of prompts (e.g., verbal prompt, model prompt, physical prompt) that are provided with the least intrusive prompt provided before the more intensive prompts [45]. The day began with a discussion of why we should teach accessible literacy to students with ESN and then a series of cycles of the trainers discussing and modeling a systematic procedure (e.g., CTD), the participants practicing the procedure in pairs or teams, and finally the participants demonstrating the procedure while a trainer watched to provide specific feedback. The second day of training included information on differentiating instruction for meeting diverse student needs such as different response modes (e.g., verbal, Augmentative and Alternative Communication [AAC, a device or support used for communication if an individual cannot speak], gestures), accessing general curriculum content, and scaffolding. The training ended with the presentation and demonstration of a commercially available grade-aligned special education curriculum, either Teaching to Standards: English Language Arts [46] for middle school or English Language Arts Grades 3–5 [47] for elementary school. Teaching to Standards: English Language Arts is a research-based curriculum that includes assistive technology supports. English Language Arts Grades 3–5 incorporates the same EBP throughout instruction but is designed for use in elementary school. The training with the curriculum was a scavenger hunt of critical features and elements within the curriculum, models by the trainers of a lesson within the curriculum, followed by practice by each participant role-playing teaching a lesson while other teachers served as students with the trainers providing guidance as needed. The special education educators- teachers and paraprofessionals- were required to implement the EBPs to teach reading comprehension skills using an adapted book provided in each curriculum at an 80% fidelity level to be approved to teach the curriculum to students.

The university team conducted the Phase 1 PD in August. All the primary participants attended the first day of training. Only the special education teachers and paraprofessionals attended the second day of training because it covered the curriculum to be used only in the special education classroom. Participants completed a workshop evaluation each day.

During the first two weeks of school, the special education teachers sent home informed consent forms for parents to review and sign. Thirty-eight students returned parent consents and were included in this study. Prior to beginning the reading comprehension instruction, each student took a pretest.

Reading comprehension instruction provided by the special education teachers and paraprofessionals began mid-September and lasted 2 months. Educators delivered the trained reading comprehension curriculum at least 30 min a day for 4 days per week as part of their school district’s mandated reading instruction. Coaching in Phase 1 consisted of face-to-face observations every two weeks of the educators implementing reading instruction with small groups of students. During each observation, the coach recorded educators’ fidelity of instructional practices (e.g., adhering to the curriculum, EBPs) with a different target student so that each student was observed over the course of this phase. After the observation, the coach and educator discussed the educator’s performance and provided additional support as needed (e.g., recommendations, modeling). At the midpoint of Phase 1, the special education educators responded to a survey to capture thoughts about the PD and coaching. At the end of Phase 1, the students took a posttest and special education teachers and paraprofessionals participated in a focus group interview about their experiences.

The goal of Phase 2 was to provide opportunities for students with ID to receive reading comprehension instruction in general education classrooms. At the beginning of Phase 2 in December, special and general educators were paired to observe each other’s classrooms. In January, the secondary participants reviewed and made recommendations to the drafted procedures and PD for educators on including students with ID in general education classrooms. Training occurred in January for the educators. The Phase 2 process included several steps. First, the special and general education teachers collaboratively planned lessons around a novel read in the general education classroom. Then, teachers co-planned pre-teaching lessons to be implemented in the special education classroom to familiarize all the students with ID with the novel and literary concepts. The university team provided an adapted version of the novel and a co-teaching lesson plan form to guide the planning and lesson delivery. Special education teachers implemented at least 4 lessons addressing literary concepts such as main idea, summarization, and character traits. Then, the teachers identified seventeen students with ID (the number of students varied by school context and decisions made by the teachers as we recommended no more than four students be included at a time in any one class so proper support could be provided) who received at least 4 lessons in the general education classroom with their nondisabled peers with the support of the special education teacher. Phase 2 ended with all students with ID completing another posttest and the special and general education teachers completing a second focus group interview and social validity survey.

2.3. Data Sources

2.3.1. Educator Surveys

Surveys were administered to teachers and paraprofessionals immediately after the initial professional development training and halfway through Phase 1. The first survey consisted of 32 five-point Likert scale items and assessed participants’ understanding of and confidence in using targeted reading practices for students with ID. The survey also contained two open-ended questions regarding additional support needed to implement the EBPs and participant overall feedback on the PD. A follow-up social validity survey contained 24 questions regarding the acceptability and usability of the coaching and inclusion practices. The survey consisted of 18 five-point Likert scale items, three items in which they checked practices they preferred and made improvements in implementing, and two open-ended questions that solicited information related to additional coaching needs or other feedback.

2.3.2. Reading Tests

Students completed a comprehension pretest prior to instruction during the first week of September and posttests at the end of Phase 1 (mid-December) and 2 (end of February). These researcher-created curriculum-based tests (see Appendix A for samples items) assessed the same reading comprehension skills using different chapters of a book that was not part of the intervention. Elementary tests contained 10 questions and middle school tests contained 13 questions (one point for each correct response) to account for additional skills taught at these grade levels. Elementary school tests addressed the following reading comprehension skill areas: vocabulary, main character, identification of the repeated story line, literal comprehension, and prediction. The middle school tests addressed vocabulary, literal comprehension, sequencing, inferential comprehension, problem and solution, main character, and main idea. Special education teachers administered the tests individually to students. Students were provided a response board of three options in which they could point to the answer choice if needed. Verbal responses also were accepted. Test items were scored as correct/incorrect.

2.3.3. Educator Fidelity

A coach collected implementation fidelity during instruction using an observation form. The number of correctly performed steps of the target practices were divided by the total number of possible steps, and then multiplied by 100 to calculate a percent of fidelity per lesson.

2.3.4. Focus Groups

The evaluator conducted an educator focus group at the end of each phase. The hour-long focus groups were held immediately after school hours using virtual conferencing software. A series of open-ended questions were posed, and educator responses were recorded and summarized. The questions for the focus groups were derived from information from the surveys and coaching observations.

2.4. Data Analysis

The ability to implement the EBPs (research question 1) was examined using the fidelity observation forms. The data collected included the number of coaching observations, number of opportunities to use either the CTD or SLP procedures, and average fidelity (including the minimum and maximum lesson fidelity) for each procedure per educator. Combining these data per educator, we calculated the average number of coaching observations, the average fidelity for each type of participant (special education teacher and paraprofessional), and the average across all participants.

Student reading comprehension skills (research questions 2 and 3) were examined using the pre- and posttest tests. A paired-sample t-test and Cohen’s d was calculated from pretest to posttest 1 and from posttest 1 (Phase 1) to posttest 2 (Phase 2) so that we could examine the student outcome of each phase of the project. Because this project included both elementary and middle school students, these statistics are reported separately by school and level and total across school levels. Descriptive statistics of raw score performance and a 2 × 3 mixed analysis of variance (ANOVA) was conducted to examine student progress by school level (elementary, middle school) across the 2 phases (pretest, posttest 1, posttest 2). Because the number of questions varied by school level, the ANOVA used percent of correct responses for a more accurate comparison by grade level.

Usability and acceptability of the PD and coaching practices from each Phase (research question 4) were assessed using information obtained from the two educator workshop evaluation surveys, two focus groups, and the social validity survey. The surveys report the frequency and percent of educator responses per question per response option. Analysis of the focus groups used a two-step process with two independent note takers during the focus groups. The two sets of notes were compared, and small edits were made so the final notes more accurately reflected comments made by the focus group participants.

3. Results

3.1. Research Question 1

EBP fidelity data summarized how well teachers and paraprofessionals implemented the EBPs during their observed lessons. Each fidelity session may have included both CTD and SLP or just one procedure, depending on the lesson.

Table 3. Fidelity data summary.

	CTD		SLP		Total
	No. Ses.	Av. %	No. Ses.	Av. %	No. Obs.	Min %	Max %
T1	7	99.7	5	96.5	436	93.4	100
T2	8	100	7	97.6	240	85.1	100
T3	10	100	14	86.0	1070	44.4	100
T4	3	100	4	93.8	361	81	100
T5	7	100	7	97.7	390	86.3	100
T6	9	98.9	9	97.4	907	85	100
T total	7.3	99.8	7.7	94.8	567.3
P1	8	100	8	93.3	541	83.6	100
P2	2	100	5	92.1	397	72	100
P3	10	100	7	82.8	225	50	100
P4	6	100	7	99.5	678	98	100
P6	5	100	4	93.1	221	78.6	100
P total	6.2	100	6.2	92.2	412.4
Av.	6.8	99.9	7	93.7	496.9

Note. T = teachers, P = paraprofessionals. CTD = constant time delay, SLP = system of least prompts, No. ses. = the number of fidelity sessions, Av. = average, No. Obs. = the number of observations.

displays the number of sessions and average percent fidelity for the CTD and SLP procedures per participant, type of participant, and overall across all participants. The Total columns display the number of individual CTD and SLP observations and the minimum and maximum fidelity across lessons per educator. The average percent of fidelity for both teachers and paraprofessionals implementation of CTD was 99.9, and there was little variation between fidelity sessions of individuals and the two different types of participants. The average percent of fidelity for teachers and paraprofessionals for SLP was 93.7. There was more variation in fidelity of individual teachers and paraprofessionals for SLP than CTD, but their average percent remained high. Although there were variations in the number of observations per participant, all educators reached 100% fidelity at least once implementing the SLP.

3.2. Research Questions 2 & 3

3.2.1. Impact of Educator Implementation of EBPs in Special Education Settings

Descriptive statistics for the pretest, posttest 1 and posttest 2 are displayed in

Table 4. Descriptive Statistics for Pretests, Posttest 1 and Posttest 2.

Assessment	School Level	% Correct	Mean	Std. Dev.	Min.	Max.	N
Pre	Elementary	58.1	5.81	2.89	1	10	21
	Middle	53.3	6.93	3.15	3	12	15
	Total	-	6.28	3.01	1	12	36
Post1	Elementary	71.0	7.10	2.0	4	10	20
	Middle	65.1	8.46	2.33	4	12	13
	Total	-	7.64	2.21	4	12	33
Post2	Elementary	81.6	8.16	1.68	5	10	19
	Middle	67.0	8.71	3.17	4	13	14
	Total	-	8.39	2.40	4	13	33

. These statistics describe the assessments prior to paired inferential testing. Prior to each inferential test the assumptions were examined and determined to have been met.

All Students’ Reading Outcome

In total, 38 students took pretest and posttest 1. There were six missing values, one missing value in pretest and five missing values in posttest 1 results. A paired sample t-test was conducted between pretest and posttest scores to examine the change in students’ achievement. The difference between participants’ pretest assessment (M = 6.13, SD = 3.06) and students’ achievement score at the posttest assessment (M = 7.59, SD = 2.23) was statistically significant t ₍₃₁₎ = 4.41, p < 0.001, Cohen’s d = 0.55 (medium effect size).

Elementary School Students’ Reading Outcome

Nineteen elementary students took pretest and posttest 1. A paired sample t-test was conducted between pretest and posttest 1 scores to examine the change in students’ achievement. The difference between participants’ pretest assessment (M = 5.68, SD = 3.02) and students’ achievement score at the posttest assessment (M = 7, SD = 2) was statistically significant t ₍₁₈₎ = 3.04, p = 0.007, Cohen’s d = 0.51 (medium effect size).

Middle School Students’ Reading Outcome

Thirteen middle school students took pretest and posttest 1. A paired sample t-test was conducted between pretest and posttest 1 scores to examine the change in students’ achievement. The difference between participants’ pretest assessment (M = 6.77, SD = 3.14) and students’ achievement score at the posttest 1 assessment (M = 8.46, SD = 2.33) was statistically significant t ₍₁₂₎ = 3.16, p = 0.008, Cohen’s d = 0.61 (medium effect size).

3.2.2. Impact of Educator Implementation of EBPs in Inclusive Settings

Student Reading Comprehension Progress Pretest, Posttest 1 and Posttest 2

Thirty students (17 elementary and 13 middle school) completed the pretest, posttest 1 and posttest 2. A one-way within-subjects design used students’ literacy growth over three assessments, pretest, posttest 1 and posttest 2. The result supports students’ literacy performance increased over time, F (2,58) = 14.27, p < 0.001, η² = 0.33. In order to find the pattern of differences in students’ literacy performance among the three assessments, post hoc pairwise comparisons were performed using Bonferroni adjustment. Students’ pretest literacy performance (M = 6.37, SD = 0.55) was significantly lower than posttest 1 (M = 7.73, SD = 0.4) and posttest 2 (M = 8.47, SD = 0.44). Students’ posttest 2 scores increased from posttest 1 (Cohen’s d = 0.34, small effect) but were not significantly higher than their posttest 1 scores (Cohen’s d = 0.55, medium effect).

The Effect of Inclusion

Out of 33 students who completed posttest 2, 17 students participated in inclusion lessons and 16 did not. To examine the effect of inclusion lessons on students’ posttest 2 scores, an independent sample t-test was conducted between students who participated in inclusion lessons and those who did not. The difference between students in inclusion lessons’ posttest 2 scores (M = 9.38, SD = 2.31) and scores of students who were not (M = 7.47, SD = 2.15) was statistically significant t ₍₃₃₎ = 2.45, p = 0.02, η² = 0.86 (large effect size).

Elementary and Middle School Difference Pretest, Posttest 1, Posttest 2

A 2 × 3 mixed analysis of variance was performed on students’ reading performance as a function of assessment and school levels (See Figure 1). The within-subjects independent variable was assessed with 3 levels (pretest, posttest1, posttest2). The between-subjects independent variable was school level with 2 levels (elementary and middle schools). The main effect of assessment was statistically significant (F (2,56) = 14.46, p < 0.001, η² = 0.34) indicating that students’ scores increased across assessments. The main effect of school level was not significant (F (1,28) = 2.22, p = 0.148, η² = 0.073) indicating no difference among elementary and middle school students. The interaction effect between assessments and school levels was not statistically significant (F (2,56) = 1.30, p = 0.281, η² = 0.044). This indicates that school level (elementary or middle school) did not impact assessment scores of posttest 1 or 2, as both groups made significant progress on the posttest.

3.3. Research Question 4

The results of the Phase 1 workshop evaluation indicated that educators’ understanding of and the perceived usability of the EBPs within various lessons and with students of different ability levels was high with a vast majority of ratings of Good or Very Good (See

Table 5. Workshop evaluation and mid-phase 1 survey results.

	After Initial Workshop					After Coaching
	None	Poor	Moderate	Good	Very Good	None	Poor	Moderate	Good	Very Good
Rate your understanding of CTD	-	-	18.8	31.3	50.0	-	-	20.0	30.0	50.0
Rate your ability to use CTD with various types of lessons (texts)	-	-	25.0	56.3	18.8	-	-	20.0	50.0	30.0
Rate your ability to use CTD with students of various ability levels	-	-	37.5	25.0	37.5	-	-	20.0	50.0	30.0
Rate your understanding of SLP	6.3		18.8	31.3	37.5			44.4	22.2	33.3
Rate your ability to use SLP with various types of lessons (texts)	-	-	31.3	43.8	18.8	-	-	33.3	33.3	33.3
Rate your ability to use SLP with students of various ability levels	6.3	-	18.8	31.3	37.5	-	-	22.2	44.4	33.3
Rate your understanding of the Example/Non-example	-	-	12.5	37.5	50.0	-	-	30.0	30.0	40.0
Rate your ability to use Example/Non-example with various types of lessons (texts)	-	-	6.3	56.2	37.5	-	-	30.0	30.0	40.0
Rate your ability to use Example/Non-example with students of various ability levels (texts)	-	-	18.8	43.8	37.5	-	-	30.0	30.0	40.0
Rate the usability of CTD data sheet	-	-	18.8	43.8	37.5	-	-	40.0	40.0	20.0
Rate the usability of SLP cards	-	-	12.5	50.0	37.5	-	-	30.0	30.0	40.0
Rate the usability of Vocabulary definitions	-	-	18.8	50.0	31.3	-	-	44.4	33.3	22.2
Rate the usability of the Sequence cards	-	-	18.8	31.3	43.8	-	-	16.67	16.67	66.7
Rate the usability of the student reader	-	-	12.5	43.8	43.8	-	-	20.0	10.0	70.0

).

Usability of the instructional materials was also rated very high. In open-ended comments, the educators reported the workshop to be informative and motivating.

Today was great, informative and inspiring as the new school year is about to begin.

It’s a great purpose and I’m honored to take part in causing positive change for students who need differentiation in learning. The team is very knowledgeable and I feel confident in getting support to participate.

Modeling of the strategies was noted to be especially helpful. Some educators reported wanting more examples of adapting general education reading content and practice (including coaching observation and feedback) with the strategies.

Seeing the modeling and hearing examples was helpful.

I enjoyed having time to practice with peers. It helped to clarify the strategy and getting direct feedback from the coaches was helpful.

I may need more practice in literacy lessons.

Different ways to model certain tasks if one way is not working effectively.

Wish I knew more about adapting content.

Table 5 also includes results from the mid-phase survey. Understanding of the CTD instructional technique and using it with various types of lessons and students with various ability levels increased after coaching. Understanding of the SLP technique and using it with students with various ability levels decreased slightly after coaching. Using SLP with various types of lessons increased after coaching. Several educators reported they needed additional practice following the workshop, and that coaching was helpful in supporting implementation, especially with the SLP procedure. A few educators indicated that taking data during instruction was difficult and that they needed support in taking data during lessons.

The continued support and feedback from Mr. [coach] has made a world of difference regarding the implementation with the Attainment curriculum.

Just need to continue to practice SLP technique

The continued feedback and support about SLP that I am receiving has been perfect.

It’s difficult to take data in that tiny little box. To be honest data is hard to take while teaching. I think I need to video tape more and look back to see where the kids struggle to give accurate data and see where my prompting is falling short. I like when we had the hand held devices to record data because it was faster.

Table 6. Social validity survey results.

	Strongly Agree	Moderately Agree	Somewhat Agree	Agree A Little	Do Not Agree
Tools in the reading curriculum helped improve my instruction	63.6	27.3	9.1	-	-
Adaptations in the reading curriculum helped improve my instruction	63.6	18.2	18.2	-	-
Models in the reading curriculum helped improve my instruction	63.6	27.3	9.1	-	-
Enjoyed getting to work with students I do not get to typically teach	100	-	-	-	-
General education students benefited from learning alongside of their peers with disabilities	66.7	33.3	-	-	-
Coaching practices were important to my teaching skills	64.7	23.5	11.8	-	-
Coaching practices improved my understanding of the purpose of EBP in inclusive classrooms during reading instruction	70.6	23.5	5.9	-	-
Coaching practices helped me implement grade aligned reading lessons using EBP	52.9	35.3	11.8	-	-
Coaching practices helped me to implement grade aligned lessons taught in general education setting	58.8	29.4	11.8	-	-
Enjoyed learning how to plan instruction with my co-teacher	47.1	35.3	11.8	5.9	-
Enjoyed learning how to deliver instruction with my co-teacher	41.2	52.9	5.9	-	-
Enjoyed building a relationship with my coach	70.6	29.4	-	-	-
Feel more comfortable approaching my coach for support	76.5	17.6	5.9	-	-
Process of coaching used in this project was helpful to me	70.6	29.4	-	-	-
My students’ learning improved as a result of being involved in this project	64.7	11.8	17.6	5.9	-
My students’ social interactions with students from the other class were meaningful	52.9	29.4	11.8	5.9	-
Students with disabilities benefited from learning alongside their same age peers	70.6	23.5	5.9	-	-

displays the ratings of all participants regarding the usability and acceptability of the Phase 2 PD and coaching in inclusive general education settings. The vast majority of educators either strongly or moderately agreed that inclusion coaching practices improved their teaching skills and that they enjoyed working together. A majority also agreed that both general education and special education students benefited socially and academically from inclusion. The first three questions on the social validity survey at the end of Phase 2 addressed the usability of the curriculum in the special education classroom. Overwhelmingly, educators rated the curriculum very high with a vast majority strongly agreeing that it helped improve their instruction.

Comments indicated that teachers needed more assistance in how to co-plan with their teaching partner, adapting grade-level texts, and pacing of lessons. Both special and general education teachers commented on how much they liked to receive the coaching and shared their excitement at the progress made by the students

I’ll need more coaching on planning with my co teacher

I need more support in adapting grade level texts

Being able to interact more with the students who visit our gen ed classroom and planning the pacing of my lesson to benefit them but also remain on pace with district expectations.

My students made AMAZING growth in both settings. I don’t have words to describe how great this program is.

The participants attended a focus group interview at the end of Phase 1. The educators indicated they could apply the EBP procedures in other subjects, and the curriculum was easy to modify for all levels of students.

I can definitely apply these techniques in my classroom in other subjects as well. Transferable.

ELA curriculum is easy to modify for all levels of students.

Comments made in the focus group supported the mid-phase survey findings indicating that extra practice during and after the training was perceived as necessary. The educators reported that students enjoyed reading the chapter books and supporting visuals and were motivated to participate in group reading instruction. Educators noted students that were previously typically unengaged appeared eager to participate in these lessons. Educators reported being surprised and pleased by students’ ability to look back at the book to find the answers and locate key vocabulary.

Students love the chapter books and reading. Students are motivated to participate group reading. We have had students that normally refuse to work who now want to participate in the lessons.

Students made new connections that I was not expecting.

Amazed by student’s skills, they look back at the book to find the answers, didn’t know they could do that.

Regarding the EBP fidelity coaching, the educators reported they liked the coaching, including modeling and feedback. They noted feedback could have been more consistent between different coaches and documented in a format they could easily reference later. One educator indicated that the coach made her a better teacher and pushed her to do more. Elementary and middle schools’ educators noted it would be helpful to see each other’s materials.

Feedback has been great from coaches. Written feedback would be helpful to help me remember what we talked about. Quick follow up email. I thought my students’ level were low and they could not do higher tasks but it was nice to have someone to encourage me to push my students a little further.

We want to see to each others’ materials and their videos.

Focus group interview comments after Phase II were centered around planning, co-teaching, the impact of inclusion on special education and general education students, and future directions. Educators reported the need for a common planning time but were able to meet after school or collaborate digitally using a shared drive for materials. Additionally, general education educators reported that the time between the initial August training and students with ID going into the general education classrooms in January was too long.

Setting a time to meet share materials helps, but also need to work on the fly whenever we see each other.

We used Google drive for sharing lesson plans helps us to save time, Email, text.

Time btw training and going into for gen ed classrooms too long for the gen ed teachers. Forgot a lot from August.

Additionally, special education teachers indicated that the pace of the general education curriculum was challenging and if a lesson did not work out, there was no time to redo the lesson and then catch up. Some teachers felt the general education curriculum was not flexible enough given what they needed to do for the students with ID, and that units may be more beneficial than individual lessons.

Pacing with gen ed classroom is hard. If lesson not work out, no time to redo the lesson and then catch up.

Gen ed curriculum not flexible enough given what they need to do for students. No time to repeat lessons.

Better to plan a unit and not just lesson so see how everything fits.

Both general education and special education educators reported to enjoy co-teaching. They described using a range of effective strategies in the general education classroom including visuals, vocabulary mapping, answering five Wh questions and summarizing, peer supports, universal behavior strategies such as hand signals, highlighted vocabulary words, repeated storylines, picture response cards, graphic organizers, turn and talk opportunities for students who were verbal, and planning lessons in advance. One strategy in particular- turn and talk- did not work for every student and often required additional supports and practice.

I loved co-teaching; helped me understand special education. I was main instructor and [special ed teacher] monitored to help both gen ed and sped students as needed. Tag team approach. I liked having a 2nd teacher in the room.

Turn and talk does not work for every student. Some do not have needed communication and/or social skills.

Educators reported to perceive that most students with ID enjoyed going to the general education classroom and interacted with general education students outside the classroom. Prior to the students with ID joining their classes, several general education educators facilitated discussions with their students regarding differences in learning and empathy for others. They also recommended the students with ID visit the general education classrooms before going in for instruction and suggested having more time together in other school situations. They also reported that the general education students would ask when the students with ID would return, saw that learning could be difficult for some students, and talked about it to their parents. In future iterations of this model, teachers should consider strategies for introducing students and learning differences in a variety of settings so they can learn about each other prior to inclusive instruction.

Most special students love going general education class, two did not like it. One got used to it, but still anxious about it. The other is apathetic about it. All gen ed love them coming. They ask when they [sped] are coming back. Social interactions going very well.

It is necessary to teach gen ed students about the needs of sped students. Hold a meet and greet before coming into classroom. More time together in other school situations would be helpful.

General ed students see that learning could be difficult for some students. Students learn that they could teach special education students (turn and talk), build their confidence. Gen ed students (peer buddies) are paying more attention because they have to teach it to someone else. They are talking about it to their parents and everyone is very excited.

4. Discussion

In this action research project, we sought to implement a PD and coaching package to support a local district’s implementation of high-quality reading instruction for students with ID. The project was implemented in two general phases involving the provision of PD and coaching support in the (a) special education class and then (b) in the general education classroom. Feedback from participating educators was sought after each phase of the project to help shape practice and resources. Overall, findings indicate the model was effective in facilitating teachers’ and paraprofessionals’ use of EBPs for reading comprehension in students with ID (e.g., CTD and SLP procedures) with high levels of fidelity. The fidelity for the educators was 99.9% for CTD and 93.7% for the SLP. This high level of fidelity is unsurprising, as many of the EBPs have been demonstrated to be effective when a range of change agents (e.g., special and general education teachers, paraprofessionals, peers, parents) are trained to use them under tightly controlled research conditions, e.g., [12,48]. We did find, however, that educators had more variability with SLP as this prompting hierarchy required the educators to make decisions about prompting levels and implementation rather than a scripted process that CTD follows every time it is implemented. If educators required additional support to maintain acceptable fidelity levels, support was provided during coaching sessions. In our project, we sought to extend the research by training educators to implement EBPs in typical school environments, using procedures employed by districts to support educators’ PD (e.g., all-day PDs, on site-coaching) using a systematic cyclical action research process.

One of the most important aims of this action research project was to improve reading comprehension outcomes for students with ID. Data indicate that participants increased their reading comprehension performance significantly during the first phase of the project and continued to make improvements, though not statistically significant, during the second phase. These outcomes are not surprising given the research on the use of EBPs in both special and general education classrooms, e.g., [49,50] to improve comprehension outcomes of student with ESN. The differences between levels of improvement between phases may be contributed to several factors. First, during the first phase of the project students likely received more intervention dosage. Special education teachers delivered reading instruction several days a week, and in small group instructional arrangements. These arrangements provided students with several opportunities to respond and receive instructional support and feedback during instruction. The instructional package was proven effective for students who were and were not fluent readers as teachers and paraprofessionals provided both a read aloud and visual supports during comprehension instruction. During the second phase, participants received instruction in the general education classroom only four times and within large group arrangements. General education teachers often spent a large portion of the lesson delivering didactic instruction which provided few opportunities for students with and without disabilities to actively engage during instruction. Second, it was unclear whether special education teachers continued intervention in the special education classrooms during this phase of the project. It is likely that some special education teachers shifted their efforts towards planning for and facilitating inclusion in the general education setting. Finally, instruction in special education settings was centered around a commercially available curriculum with scripted research-supported instructional routines, whereas in the general education classroom, teachers delivered instruction based on their planning. These teacher- planned lessons often were composed of different elements than the scripted lessons. It is worth noting that the social validity item with the most variability in responses (see Table 5 agree a little to strongly agree) related to enjoying learning to plan with a co-teacher. In the future, we plan to further support co-planning and the reading intervention in both special and general education settings during the second phase of the project. We noted that one of the challenges of the general education settings is that teachers moved through the curriculum at a rapid pace that may not provide some learners with and without ID sufficient opportunity to acquire critical content. We suggest that students learning in the general education classroom might be enhanced by additional exposure to the general education content and specialized instruction.

It is important to recognize the contributions that occurred as a result of using action research methodology. The model that resulted from the action research was clearly successful in impacting the educators use of EBP and students learning outcomes. The components of the EBP PD and coaching influenced educators’ fidelity of the EBPs. The success of the project was influenced by the engagement of the educators and district during the study. By working together to solve a local problem, educators were invested in the research process to close the research to practice gap. While this process can be tedious and time consuming for practitioners, educators in this project were research partners with a voice in how the model was designed and implemented to not only promote their success but the success of the students with ID in both settings. Educators reflected formally during focus groups and informally during coaching sessions on their own practice. For some paraeducators, the project was the first time they were asked to reflect on their teaching and view themselves in videos teaching. Both sets of partners were responsive to the research and the practical needs via the action research approach.

Finally, the action research process provided the researchers with invaluable insights about how to improve the model during the implementation for these educators and in future implementations with new educators [51]. For example, feedback provided by educators regarding the timeliness of PD and the delivery of the strategies informed the next iteration of the timeline for the model. Additionally, we gained ideas during the focus groups about how to better build student to student and general educator to student relationships prior to the general education lessons. The EBP PD and coaching were significantly influenced by the participatory nature of this convergent mixed methods study.

4.1. Implications for Practice

In the first phase and immediately following the initial group PD, special education teachers and paraprofessionals reported high levels of confidence in their ability to implement the EBPs. Interestingly, 6 weeks after the PD and opportunities to implement the procedures, several educators reported lower levels of confidence. This change in some participants’ responding may have reflected their initial lack of experience in implementing the EBPs in natural contexts. The Phase 1 change in educators confidence levels may also reflect a contrast in training and implementation conditions. During our initial training, we presented opportunities to practice in simulated one-to-one instructional arrangements with adults playing the role of a student. During classroom implementation, educators often presented instruction in small group arrangements with learners that required additional supports to facilitate engagement (e.g., communication supports, attentional cues, positive behavior supports). A disconnect between training and implementation conditions has been cited as a factor in the failure of many PD to produce change in the classrooms [52,53,54]. In future iterations of this model, we will consider strategies for more closely connecting training conditions to those of the classroom (e.g., instructional arrangements, student needs).

Another potential variable related to changes in educator levels of performance and confidence may have been the latency (i.e., several weeks) between initial training and the implementation of reading instruction. We conducted training during the summer, immediately prior to the school year starting, as to not disrupt educators’ beginning of the school year routines. Once the school year started, we then obtained parental consent, conducted screening procedures, and administered pretest measures prior to the implementation of reading instruction in the special education classroom. The gap between training and implementation was even more pronounced as educators entered Phase 2 of the project, which was implemented in the general education classroom. General education teachers reported that they had forgotten much of the original content presented in the initial training. Despite potential contributions to teachers’ workload, we recommend that future action research consider creative ways to deliver training once school is in session. As research has demonstrated both online and face to face PD are effective for educator and student performance [55], this might involve the development of a hybrid training model, in which educators receive didactic instruction using digital training materials and then meet (e.g., after school, during a portion of the school day) with coaches to rehearse the instructional strategies.

In Phase 2 of the project, both special education and general education educators provided support to students with ID in the general education classroom. Interestingly, we also observed peers without ID interacting and offering support to included students. Interactions between students with and without disabilities, whether initiated by educators or by students, is not uncommon in inclusive [56]. It may be pertinent to prepare for peer engagement from the onset as students are social by nature and as we found, did not wait for training to engage with their peers with ESN.

Despite observed success in including students with ID during general education literacy lessons, teachers reported challenges in adapting the general education curriculum including (a) understanding expectations and accessibility needs for students with ID, (b) a lack of time to plan with colleagues, and (c) a lack of time to prepare and implement differentiated instruction. Finding a common time for planning for educators is crucial if inclusive opportunities are to be successful for all participants [57,58]. Additionally, pacing of the content in the general education classroom was reported as a challenge for both the special educators and students. We found that unit planning tools were required with targeted prioritized content and lesson plans designed for both classrooms to include repeated instructional opportunities for students to engage [45].

4.2. Implications for Research

For research purposes, it would be beneficial to test the training conditions under which educators were trained and coached to use the two strategies. As research on PD has shown (ADD), PD in isolation is not as effective PD with follow up experiences but it may be necessary to test the length of time between the PD and follow up on educators’ confidence and fidelity for the practices. An additional related area for consideration within research based upon our findings is the use of role play during acquisition of strategies versus conditions for implementation with students with ESN. As our educators found it more challenging to implement the strategies to small groups of students even though they had demonstrated the strategy to fidelity at the end of training, it may be worthwhile to test a variety of role play conditions that result in the most similar fidelity outcomes upon educator implementation of the strategy with students.

Fortunately, our package also included on-site coaching in which a member of the research team met with teachers and paraprofessionals, observed their delivery of instruction, and provided feedback. Several participants indicated that coaching components of the package were essential during implementation. They reported a need for additional practice following the initial training and support in differentiating instruction for individuals’ students in general education settings. For example, participants noted that feedback was inconsistent across coaches. Some participants suggested that coaching might be improved by ensuring consistency of support between coaches and providing written feedback to the participants following each coaching session. To address these participant concerns, we adopted and used a consistent coaching protocol that included providing documentation of performance feedback and action steps. Formal evaluation of the coaching dynamic (i.e., the content of and supportive nature within coaching across coaches through the use of a systematic protocol versus a more individualized approach) may be warranted.

An additional area that may be beneficial for further research would be ways to address the dosage of instruction required per literary concept within the special and general education classroom. As the pacing of the lessons and content coverage within the general education classroom typically required additional pre and post instruction lessons with explicit instruction within the special education classroom, it would be important to determine what dosage or amount of explicit instruction would maximize student learning of the content and participation in the general education lessons.

5. Conclusions

Hine [43] stated “Ultimately, the solutions-based focus, emphasis on fostering practitioner empowerment, and pragmatic appeal of action research collectively render this research methodology a worthwhile professional development activity for teachers.” (p. 161). The current project exemplified the use of action research to develop and refine a model of PD that included curricular supports and multiple coaching sessions. The model was designed to improve educator fidelity of implementing EBPs within instruction and student reading comprehension outcomes in separate and inclusive settings. The action research process involved regular evaluation and feedback sessions with experts, district administration, and educators. Even as educators and students experienced significant improvements, their feedback influenced revisions to the model to better address identified needs and usability issues. More importantly, the partners and participants appreciated having a voice to refine the model and see their ideas put into action. The project exemplified action research by using a scientific approach to solve a practical problem within the district. While barriers within the model were identified during the cycle such as collaborative planning time and preparation for students with ID for the general education classrooms, the potential for improvement and opportunities for additional staff and students is apparent.