Student Attendance Patterns as Actionable Early Warning Indicators of High School Graduation Outcomes: Findings from an Urban Alternative Charter School

Abstract

Background: Students who fail to complete high school, on average, have less earnings, pay fewer taxes, and require increased government expenditures compared to those who do reach this milestone. Especially in urban jurisdictions, this can lead to reduced municipal fiscal health. Alternative high schools have been one intervention employed to improve the outcomes of students who have previously dropped out. Also, in recent decades, early warning indicator (EWI) systems have been put in place to flag students who are at risk for not completing school. However, the current EWI metrics for student attendance are insufficient for the population that attends alternative high schools. Methods: Administrative data were obtained from an alternative charter high school in a large urban city in the northeast United States (n = 224). Logistic regression analyses were conducted to test the utility of traditional EWI operationalizations of student attendance against more targeted measures, with student graduation serving as a dichotomous outcome variable. Results: Of the models tested, the model that flagged student non-attendance as missing consecutive days three or more times during the first 12 weeks of school had the greatest explanatory power (McFadden’s R² = 0.301) and best overall model fit. Conclusions: Traditional definitions of attendance are less useful in schools doing re-engagement work, and more targeted indicators serve as more effective EWIs in these settings.

1. Introduction

The economic importance of education has been well documented over many decades [1,2,3,4,5,6,7,8]. Psacharopoulos and Patrinos [9] conducted a review on education’s return on investment based on the literature from 139 countries over a 60-year span and found strong evidence that increased schooling leads to increased earnings. There are large societal costs when individuals fail to complete high school. Individuals who do not complete high school are less apt to participate in the economy, pay taxes, or vote [10]. As a result, the economic health of larger cities is largely influenced by how educated its citizens are. Education level matters tremendously on an individual level as well. Those who fail to complete high school are more apt to engage in crime, become incarcerated, consume government resources, and experience adverse health outcomes than those who earn a diploma [10,11,12]. These individuals are also less likely to be employed, have shorter life spans, and live in poorer health on average than those who do [10,12,13]. As such, it is important to find ways to identify students who are at risk of not graduating for intervention.

1.1. Student Attendance and Graduation Outcomes

Student attendance has been linked to graduation outcomes [14,15,16,17,18,19,20,21,22]. This makes sense; students who are present more often in school are more likely to reap the benefits of instruction. Rumberger [10] describes attendance as “one of the most direct and visible indicators of engagement” (p. 169). In his review of 19 studies, 13 had findings suggesting that high levels of absenteeism led to an increased likelihood of dropping out of high school [10]. Survey research conducted in Philadelphia found that more than half of participants—all of whom dropped out of high school—reported factors outside of school contributing to why they did not attend school [23]. Patterns of poor high school attendance often appear well in advance of a student dropping out of school.

Historically, studies have explored the relationship between student attendance and high school graduation outcomes in one of two manners: (1) as a percentage of days attended [24]; or (2) in terms of the raw number of days either attended or missed over a given time frame, typically the span of an entire school year [14,19]. Cratty [15] found that approximately one in three students who misses 15 or more days fails to graduate from high school; this increases to about half of students after 21 or more days have been missed. Several studies relied on the National Educational Longitudinal Study (NELS), which asks teachers to rate their students’ attendance over a four-week period on a four-point scale [20,21,22]. While each of these studies found attendance to significantly predict graduation outcomes, the use of NELS introduces measurement issues. Reliance on self-report data could introduce issues related to faulty recollection or social desirability [25]. Also, measuring attendance over a four-week period (approximately one-tenth of a typical school year) might not be representative of a full academic year. Overall, regardless of how student attendance has been operationalized, it consistently been found to be a significant predictor of high school graduation status.

1.2. Early Warning Indicators

Over the past couple of decades, researchers have developed early warning indicators (EWIs) aimed at identifying students who are at risk of not graduating in hopes of being able to intervene [26,27,28,29]. These EWIs tend to focus on course grades, student attendance, and behavioral markers (i.e., suspension from school). Scholars have arrived at different ways of operationalizing poor attendance, but most identify this as a student who misses at least two to four weeks of school over the course of an academic year. Mac Iver and Messel [17,30] have operationalized this as missing 10% of the school year. Given that a typical school year in the United States is 180 days, their attendance EWI is missing 18 days or more. This metric was subsequently codified in the Every Student Succeeds Act passed by the U.S. Congress in late 2015 [31]. Others have operationalized this as missing 14 days [32] or 15 days [15,18,19,33]. Balfanz et al. [27] and Neild et al. [24] operationalized poor attendance as being present fewer than 80% of school days, which in a 180-day school year would constitute more than 36 absences. In each of the studies listed here, the student attendance EWI significantly predicted high school graduation outcomes. Students who triggered the EWI with their attendance patterns were less likely to earn a high school diploma.

1.3. Issues with Attendance EWIs

Alternative high schools have been one mechanism to improve graduation outcomes for students deemed to be either ‘at risk’ and/or those who have previously dropped out of school [10]. Dropping out of school is rarely the result of a single event; rather, it is a process of gradual disengagement from school [34]. Students who opt to re-engage and re-enroll in school often find developing the habit of regularly attending school again to be challenging [35]. While EWIs that focus on student attendance have historically been quite useful in traditional school settings, they are potentially less efficacious for alternative high schools settings. However, this is an area that is understudied. The current study sought to explore this and was guided by the following research questions:

• To what extent do traditional EWI operationalizations of student attendance successfully predict high school graduation outcomes in an alternative high school setting?
  • Missing more than 10% of days
  • Missing more than 20% of days
• To what extent do targeted operationalizations of student attendance successfully predict high school graduation outcomes in an alternative high school setting?
  • Missing consecutive days in September
  • Missing consecutive days three times in the first 12 weeks
  • Missing 4 or more days in September

2. Materials and Methods

The current study explores quantitative early warning indicators of student attendance at Northeast Charter School (NCS; a pseudonym). NCS is an alternative charter high school located in a large, high poverty city in the northeast region of the United States that exclusively enrolls students who previously dropped out of school. Students who attend NCS have both academic and vocational requirements. Those who attend through completion earn a high school diploma, are paired with a trade, and earn professionally recognized certification(s). The vocational tracks available for students include childcare, health care, and environmentally conscious construction, which includes solar panel installation. The school holds a graduation each August. Students who are close to graduating but have not completed all of their requirements in time for the August graduation ceremony are eligible to graduate in a smaller ceremony in November. The school enrolls fewer than 250 students each year and has a low teacher-to-student ratio. This research was approved by Auburn University’s Institutional Review Board (#18-230 EP 1807).

2.1. Participants

Administrative data were obtained for each student enrolled at NCS during the 2018–2019 school year. Just under half (44.59%) of students graduated in August; an additional 16% of students graduated in November. There were slightly more female (53.15%) than male (46.85%) students, and the overwhelming majority of students were Black or African American (91.74%). More than half of the participants received government benefits (65.29%), a proxy for low socioeconomic status. Approximately one-fourth of the participants were either pregnant or were a parent (25.23%), and about the same proportion had a parent who had been incarcerated (24.85%). Just over one-third of participants (36.69%) had been arrested themselves. About one in five had been homeless at one point in their lives (18.93%), and about one in eight had been in foster care at some point (12.43%). See

Table 1. Demographics of study’s participants.

Variable	N	%
Graduated in Aug	99	44.59
Graduated in Nov	135	60.81
Gender
Female	118	53.15
Male	104	46.85
Race/Ethnicity
American Indian/Alaskan	7	3.21
Black/African American	193	91.74
Hawaiian/Pacific Islander	1	0.46
Hispanic	9	4.13
White	3	1.38
More than one race	3	1.38
Parent/Pregnant	55	25.23
Receives government benefits	111	65.29
Was ever in foster care	21	12.43
Was ever homeless	32	18.93
Was ever arrested	62	36.69
Parent was ever incarcerated	42	24.85

N = 224.

for demographics of this study’s participants.

2.2. Description of Models Tested

A series of logistic regression models were tested to understand the extent to which previous operationalizations of student attendance successfully predict graduation outcomes in an alternative charter high school. The study also sought to learn how well more targeted operationalizations of high school graduation similarly predicted graduation from high school. Each of the models tested contained the same dichotomous graduation outcome. The first five models tested operationalized graduation as having earned a diploma by the August graduation ceremony. The same five models were tested again, this time operationalizing the outcome as having earned a diploma by the November ceremony.

Each of the models contained a single predictor variable that measured student attendance in one of five ways: (1) missing more than 10% of school days; (2) missing more than 20% of days; (3) missing consecutive days during September, the first month of school; (4) missing consecutive days on three occasions during the first 12 weeks of the school year; and (5) missing four or more days in September. Each model also contained a vector of covariates to control for societal factors that have been associated with a reduced likelihood of graduation including the following: (1) being male; (2) pregnant or a parent [36]; (3) receiving government benefits [37]; (4) having ever been in foster care [38]; (5) having ever been homeless [39]; (6) having ever been arrested [12]; and (7) having a parent who is incarcerated [40]. The literature on the relationship between student gender and high school graduation outcomes is mixed; however, in every study in which student gender was a significant predictor of either graduation or dropout, males had lower odds of graduating [10,18]. Each of the models testing attendance EWI predictor variables are as follows:

$$\log \left(\frac{P(Y=1)}{1-P(Y-1)}\right)={\beta }_0+{\beta }_1{X}_1+{\beta }_2{X}_2+\dots +{\beta }_n{X}_n+\epsilon$$

where P(Y = 1) is the probability that the participant graduated from high school, ${\beta }_0$ is the intercept, ${\beta }_1,{\beta }_2,\dots ,{\beta }_n$ are the coefficients for the predictor variables, $X_1$ is the student attendance EWI for the model, $X_2,\dots ,X_n$ are the covariates, and $\epsilon$ represents the error term.

2.3. Data Analysis

All the models were tested using logistic regression analysis [41], and all analyses were conducted using Stata version 17. Prior to the analyses being conducted, the data were screened to ensure that the requisite assumptions were met for logistic regression analysis. Logistic regression does not require multivariate normality or homoscedasticity, nor does it assume a normal distribution of error terms [42]. Collinearity diagnostics were run for each model, and variance inflation factor values were found to be acceptable for each of the models tested. Tabachnick and Fidell’s [42] guidance for sample size is that it should be greater than 50 + 8m, where m represents the number of variables included in the model. In this study, each model tested included one student attendance variable, along with seven additional covariates. While this study’s sample is not particularly large (n = 224), it meets this requirement. The models were evaluated in terms of model fit by comparing log pseudolikelihood values [43], AIC [44] and BIC values [45], as well as in terms of an approximation of the amount of variance explained by the predictor variables included and the percent of cases that are correctly classified [46].

3. Results

Two series of models were tested to learn the utility of five different ways of measuring student attendance as an EWI. Models were tested for both August and November graduations; results for each were similar. See

Table 2. Student attendance variables included in each model.

Model	Title 2
1	90%+ attendance
2	80%+ attendance
3	Missing consecutive days in Sept.
4	Missing consecutive days 3× in 12 wks
5	Missing 4 or more days in Sept.

and

Table 3. Student attendance and model fit indices.

		Model 1 a	Model 2 a	Model 3	Model 4	Model 5
August Graduation
Attendance	OR(SE)	#	23.83 (24.85) **	0.223 (0.082) **	0.086 (0.034) **	0.173 (0.066) **
Model Fit	McFadden’s R2	0.058	0.158	0.142	0.264	0.165
	−2 LPL	−101.865	−98.010	−99.991	−85.708	−97.229
	AIC	219.73	214.200	217.982	189.416	212.458
	BIC	244.129	242.369	246.152	217.585	240.627
	Correct Classification	58.33%	62.13%	70.41	77.51%	72.19%
November Graduation
Attendance	OR(SE)	#	#	0.217 (0.083) **	0.061 (0.030) **	0.182 (0.070) **
Model Fit	McFadden’s R2	0.083	0.075	0.172	0.301	0.191
	−2 LPL	−93.441	−88.141	−88.883	−74.995	−86.868
	AIC	202.883	192.282	195.769	167.991	191.735
	BIC	227.281	215.929	223.936	196.160	219.904
	Correct Classification	67.95%	64.08%	72.78%	78.70%	73.37%

Note: ** p < 0.01; * p < 0.05; # attendance variable perfectly predicted nongraduation; ^a model was not significant improvement over intercept-only model; each model included a vector of covariates including (1) gender, (2) parent/pregnant, (3) received gov’t benefits, (4) ever been in foster care, (5) ever been homeless, (6) ever been arrested, and (7) parent is incarcerated.

for the odds ratios and standard errors for each operationalization of student attendance, as well as for model fit indices for each. See

Table A1. Student Attendance and Model Fit Indices—August Graduation Models.

	Model 1 a	Model 2 a	Model 3	Model 4	Model 5
Variable	OR (SE)	OR (SE)	OR (SE)	OR (SE)	OR (SE)
Male	1.399 (0.494)	1.185 (0.429)	1.406 (0.505)	1.062 (0.431)	1.494 (0.546)
Parent or pregnant	0.197 (0.223)	0.204 (0.231)	0.214 (0.255)	0.254 (0.319)	0.153 (0.178)
Receives gov’t benefits	1.154 (0.419)	1.007 (0.370)	1.023 (0.375)	1.033 (0.424)	1.063 (0.396)
Ever in foster care	1.312 (0.689)	1.339 (0.710)	1.414 (0.778)	1.304 (0.785)	1.269 (0.701)
Ever was homeless	0.346 (0.155) *	0.410 (0.183) *	0.351 (0.160) *	0.425 (0.214)	0.396 (0.184) *
Ever was arrested	0.753 (0.275)	0.759 (0.288)	0.672 (0.253)	0.576 (0.242)	0.697 (0.266)
Parent was incarcerated	1.091 (0.435)	0.976 (0.409)	1.029 (0.428)	1.380 (0.636)	1.078 (0.457)
Attendance EWI	#	23.83 (24.85) **	0.223 (0.083) **	0.086 (0.0340) **	0.173 (0.066) **
McFadden’s R2	0.058	0.158	0.142	0.264	0.165
−2 Log Pseudolikelihood	−101.865	−98.010	−99.991	−85.708	−97.229
AIC	219.73	214.200	217.982	189.416	212.458
BIC	244.129	242.369	246.152	217.585	240.627
Correct Classification	58.33%	62.13%	70.41%	77.51%	72.19%

Note: ** p < 0.01; * p < 0.05; # attendance variable perfectly predicted nongraduation; ^a model was not significant improvement over intercept-only model; the outcome variable was a binary categorical variable indicating whether the student graduated by the August graduation (1 = Yes; 0 = No). The reference category was a student who was not male, not pregnant or a parent, did not receive government benefits, was never homeless or in foster care, had never been arrested, and did not have a parent who was incarcerated. See Table 2 for the manner in which attendance was operationalized for each mo.

and

Table A2. Student Attendance and Model Fit Indices—November Graduation Models.

	Model 1 a	Model 2 a	Model 3	Model 4	Model 5
Variable	OR (SE)	OR (SE)	OR (SE)	OR (SE)	OR (SE)
Male	1.418 (0.538)	1.217 (0.473)	1.350 (0.527)	1.038 (0.462)	1.470 (0.586)
Parent or pregnant	0.232 (0.219)	0.240 (0.227)	0.265 (0.273)	0.314 (0.299)	0.182 (0.179)
Receives gov’t benefits	0.513 (0.204)	0.454 (0.182) *	0.446 (0.185)	0.342 (0.160) *	0.448 (0.189)
Ever in foster care	0.786 (0.432)	0.807 (0.182)	0.823 (0.478)	0.719 (0.418)	0.719 (0.418)
Ever was homeless	0.336 (0.146)*	0.390 (0.170) *	0.343 (0.155) *	0.386 (0.177) *	0.386 (0.177) *
Ever was arrested	0.518 (0.200)	0.507 (0.205)	0.445 (0.181) *	0.330 (0.152)	0.466 (0.191)
Parent was incarcerated	1.334 (0.575)	1.277 (0.573)	1.427 (0.652)	1.991 (0.993)	1.416 (0.651)
Attendance EWI	#	#	0.217 (0.083) **	0.061 (0.030) **	0.182 (0.070) **
McFadden’s R2	0.083	0.075	0.172	0.301	0.191
−2 Log Pseudolikelihood	−93.441	−88.141	−88.883	−74.995	−86.868
AIC	202.883	192.282	195.769	167.991	191.735
BIC	227.281	215.929	223.936	196.160	219.904
Correct Classification	67.95%	64.08%	72.78%	78.70%	73.37%

Note: ** p < 0.01; * p < 0.05; # attendance variable perfectly predicted nongraduation; ^a model was not significant improvement over intercept-only model; the outcome variable was a binary categorical variable indicating whether the student graduated by the November graduation (1 = Yes; 0 = No). The reference category was a student who was not male, not pregnant or a parent, did not receive government benefits, was never homeless or in foster care, had never been arrested, and did not have a parent who was incarcerated. See Table 2 for the manner in which attendance was operationalized for each model.

for complete results, including the odds ratios and standard errors associated with covariates in each model.

3.1. Traditional Attendance EWIs

Two traditional attendance metrics were tested. Overall, these EWI metrics for measuring student attendance were not found to be particularly useful for a population of students who were re-engaging in school and attending an alternative high school. The EWI that flags student attendance when 10% of days were missed lacked appropriate levels of sensitivity for this sample as every student who missed 10% of days or more also failed to graduate. This was true when the outcome variable measured student success as having graduated in the August or November ceremonies. The McFadden’s R² values indicate that both models tested using this attendance EWI explained less than 10% of the overall variance. The August graduation model only correctly classified cases 58.33% of the time.

When student attendance was operationalized as missing 20% of days or more, it yielded a very large odds ratio (23.83) for graduation by August; this is because 29 of the 30 students in the sample who had an attendance of 80% or better graduated. While this model explained approximately 15% of the variance, it only correctly classified cases 62.13% of the time. When the outcome variable was operationalized as having graduated in the November ceremony, all of the students who missed 20% or more days failed to graduate. This model correctly classified cases 64.08% of the time. For this student population, traditional ways of flagging students as “at risk” of not graduating lack an appropriate amount of sensitivity. None of the four models tested accurately classified cases better than 70% of the time.

3.2. Targeted Attendance EWIs

Each of the models that used targeted operationalizations of student attendance performed better than the traditional EWI models. The simplest of these models operationalized attendance as having missed consecutive days in September, the first month of the school year. Compared to students who did not miss consecutive days in September, students who triggered this EWI, the odds of graduating were between 21 and 23%. The models explained between 14.2 and 17.2% of the variance and correctly classified classes between 70 and 73% of the time—an improvement over the traditional attendance EWIs.

The model that performed best operationalized attendance as having missed consecutive days three or more times during the first 12 weeks. The attendance variable itself yielded a significant odds ratios of 0.086 (for August graduation) and 0.061 (for November graduation), indicating that students who missed consecutive days three times or more over the first 12 weeks of school had a less than 10% of the odds of graduating compared to a student who did not miss consecutive days as often. Overall, the model fit indices suggest that this model performed best among those tested, as evidenced by the AIC and BIC values, which were lowest for this model. McFadden’s R² was greatest for this model (0.264 for August graduation; 0.301 for November graduation). The model also correctly classified cases the most often (77.51–78.70%).

The final set of models operationalized student attendance as having missed four or more days in September. These models performed similar to the first targeted EWI described—having missed consecutive days in September. Compared to students who missed three or less days in September, for those who missed four or more days in the month, the odds of graduating were approximately 17–19%. The models explained about 16–18% of the overall variance and correctly classified cases 72–74% of the time. Overall, the results suggest that more targeted, actionable EWIs might be of greater use for this population of students.

4. Discussion

Improving high school graduation outcomes is a matter of great concern, especially in American cities. The advent of EWI systems and data-driven student intervention has led to great improvements in graduation outcomes in the first two decades of the 21st century [10,18]. Alternative high schools designed to re-engage students who previously dropped out have also been an intervention used to improve graduation rates. While ample literature has explored the efficacy of EWIs [30], almost all of it has been conducted in traditional schooling contexts. The current study aimed to add to the literature on EWIs, testing traditional attendance EWIs in non-traditional school settings (i.e., an alternative high school), as well as more targeted attendance EWIs. When the models were run using August graduation as the dependent variable, two of the targeted measures that looked exclusively at attendance patterns during the first month of school performed similar to the less than 80% attendance measure in terms of the amount of variance explained by the model. However, it is plausible that if students were subjected to interventions during the first month of school, that they might have arrived at an overall attendance greater than 80%. The model that measured attendance as having missed consecutive days three or more times during the first 12 weeks was the most useful and had the best model fit along multiple measures. This is line with the literature suggesting that disengaging from school is a process [34,47]. Students who fall into the habit of missing consecutive days are more likely to eventually decide to disengage altogether again.

The advent of EWIs and their use in schools has been one of the better examples of data-driven decision-making in U.S. schools in the current century, and those who have undertaken work in this area should be applauded. However, especially when working with a more ‘at risk’ population that includes students who have previously dropped out of school, waiting for a student to miss 18 days (10% of a traditional 180-day school year) or 36 days of school (20%) before intervening seems like a missed opportunity. Previous work has found that more targeted operationalizations of course grades (i.e., looking at individual marking periods rather than the entire school year) can successfully predict graduation outcomes [18]. Doing the same with attendance, especially for students who have previously struggled with regular attendance, has great potential as well. It is also possible that attendance EWIs might need to be revisited in the post-COVID-19 pandemic era, as chronic absenteeism has become an issue of major concern in the United States [48,49] and elsewhere [47,50].

The finding that more targeted EWIs of attendance are more useful with this population is promising. Having more actionable EWIs for schools to use to intervene early on creates the potential for better long-term outcomes for students, and for society at large. When a city’s youth become more educated, more tax dollars can be devoted to municipal improvement efforts and to better fund education, and fewer dollars need to be allocated to health care for those in poverty and to the criminal justice system [10]. Efforts aimed at improving graduation outcomes can make a substantial difference for the individual student; however, they can also make a difference for the rest of that jurisdiction’s citizens as well.

4.1. Implications for Urban Economies

Educational interventions that can increase graduation outcomes and create a better-educated citizenry have important implications for local economies. This is particularly true in high-density urban areas, which in the United States have historically had higher concentrations of individuals living in poverty [51], which in turn, generally translates to increased outlays in public expenditures and fewer tax revenues collected [10,52,53]. Further, this is of particular importance for urban economies, which often have more government-owned property that is generally not taxable [54]. The current study’s findings suggest that targeted EWI metrics have the potential to improve educational outcomes, which subsequently can lead to more sustainable urban fiscal policy. Interventions that can improve educational outcomes for a city’s youth can contribute to the long-term health of a city, given that individuals who are better educated will earn more, cost the city less in services, and pay more taxes [10]—all of which affords a municipality the latitude to invest more in areas like education.

4.2. Limitations and Future Research

There are limitations with this work that are worth noting. First, this study’s sample was drawn from a single high school in a single urban municipality in the United States. More work would need to be undertaken to see if these findings hold in other settings and with other samples. It is also worth noting that the covariates that were controlled for in this study were all from self-report survey data collected by the school. Finally, this study tested the viability of three more actionable, targeted, short-term measures of student attendance as predictors of high school graduation. It is possible, even likely, that there are additional better short-term measures of attendance. Future work should continue to explore this, especially with machine learning. Kartini et al. [55] recently conducted a study which used a range of machine learning approaches, primarily using student course grades as predictors to model private school graduation outcomes in Indonesia. Similar approaches could be employed to predict graduation outcomes using student attendance in the United States and elsewhere.

5. Conclusions

This study sought to better understand the utility of traditional early warning indicators for student attendance in an alternative high school setting where the student population is comprised of those who have formerly dropped out of school. It was hypothesized that traditional EWIs might not be as useful for this population, and this was at least partially confirmed. Each of the three models that used more targeted, short-term measures of student attendance performed better than those using traditional attendance EWIs in terms of their ability to correctly classify cases.