**3. Results**

The first results in Table 1 showed a trend comparison over the two months of the project, from the year before the intervention to the first year of implementation. During the two years under investigation here, the FAFSA application opened on 1 January; the figure shows that just a month into the filing season, 381 students already submitted their applications in 2012, and 422 submitted their applications during the year Say Yes announced the tuition guarantee and the FAFSA Completion project began. By February 15, a few of the applications submitted resulted from the project. The computer lab sessions in Phase I were already completed by then, but only students who qualified as independent could submit during those sessions. In both years, the scholarship fair in phase II was held by the end of January, and the fair accounted for nearly a third of submitted applications by 15 February.

**Table 1.** The difference in Differences Analysis Comparing Buffalo to Other Urban Centers, 2011–2013.


\*\*\* *p <* 0.001, \*\* *p <* 0.01, ~ *p <* 0.10.

Phase III began during the week of 22 February, when we expected more significant gains in FAFSA application submissions. For the next month, the rate of increase during the project exceeded the rate from 2012. During March, the rate of increased FAFSA submissions grew by 27% in 2012 and more than 35% in 2013. By the submission deadline for Say Yes to Education, the number of FAFSA applications submitted to the U.S. Department of Education grew nearly 62%. These data alone do not indicate whether the increase is attributable to the tuition guarantee, the FAFSA Completion project, or other factors. However, it does suggest a substantial increase that is worth exploring further.

There is another difference that we note descriptively, which is not pictured in the figure. FSA provided us with two numbers—FAFSA submitted and FAFSA Completed. The difference between the two numbers was that some applications were rejected because they either included errors or were missing signatures. If we take the ratio of completed applications to those submitted, we have an accuracy rate. In 2012, the federal government accepted approximately 87% of all applications submitted by BPS students. During the 2013 year, when the project was underway, those accuracy rates improved to 92%. We did not see accuracy rates improve across the other schools in this study. While we did not focus on this outcome, it may be most directly linked to the work of the FAFSA Completion project because individualized help was more likely to result in catching errors before the applications were submitted. In practical terms, that may amount to as many as 50 students in BPS schools whose applications were accepted that would not have been without the assistance of the project.

The first contrast compares outcomes for BPS students to those in Albany, Syracuse, or Rochester public schools. Table 1 summarizes the regression analysis, and the findings

can be understood relative to the total number of FAFSA forms submitted at a given school each year. Approximately 58 students submit the FAFSA each year across each high school included in the study. Schools with a higher enrollment of low-income students—as approximated by free or reduced lunch (FRL) eligibility—report a slightly lower average. A 10-percentage point decline in FRL would result in an average of five additional students completing the FAFSA. The difference-in-differences coefficient is an interaction term between year (pre- and post-program intervention) and treatment location (Buffalo vs. other three metro areas). The result compares BPS schools in the treatment year to BPS before the intervention and the three comparable mid-sized cities, which we expect to have a similar trend over time. The DID coefficient suggests that BPS schools saw an increase of 30 FAFSA applications submitted, compared to the trend for the other metro areas an increase of approximately 50%, slightly less than the descriptive difference shown in Figure 1 above.

**Figure 1.** Comparing FAFSA Completion Rates in Buffalo, 2012–2013.

The second analysis in Table 2 compares BPS schools to the Buffalo public charter schools, which were eligible for the tuition guarantee but did not receive support from the FAFSA completion project during the first year. The sample size for the second analysis was much smaller because there are only sixteen BPS schools and five charter high schools in Buffalo, which accounts for the more modest *p*-values reported here. The mean number of applications submitted across all schools was 25, which was considerably lower than in the last analysis—and the reason is simple. The metropolitan areas outside of Buffalo have fewer high schools but are more extensive. A similar relationship exists between free and reduced lunch eligibility, indicating a 10-percentage point decrease in the free or reduced lunch eligibility would result in an additional eight students submitting their forms. The DID coefficient in this analysis suggests that BPS schools nearly doubled their FAFSA completion numbers from the first to the second year compared to the charter schools. The DID coefficient suggests that controlling for other school characteristics, an additional 24 students per school completed their FAFSA in Buffalo schools.


**Table 2.** The Difference in Differences Analysis Comparing BPS Schools to Buffalo Charter Schools, 2011–2013.

\*\*\* *p <* 0.001, \*\* *p <* 0.01, \* *p <* 0.05, ~ *p <* 0.10.

The analyses reported above provide evidence to suggest two key outcomes. First, it appeared that the announcement of the tuition guarantee influenced students' FAFSA submission behavior, which should not be a surprise given that it is a condition of eligibility. When comparing BPS to other mid-sized cities in NY, Buffalo increased the number of students completing the FAFSA, even after controlling for key demographic characteristics of schools. In the descriptive analysis, we reported a change of 62% from the year before the announcement of the Say Yes tuition guarantee to the first year of implementation. The comparison with Syracuse, Rochester, and Albany indicated a 50% increase after controlling for other factors, which was similar in magnitude. The comparison with the public charter schools in Buffalo may indicate the independent effects of the FAFSA completion project because both groups were eligible for the tuition guarantee. However, only BPS received the FAFSA completion support.

It is important to recognize that the differences reported here were by April 12 of that year. We know that in the past, many students would have completed their FAFSA in August, right before the start of college—and many of them were attending community colleges. The advantage of setting the deadline for submission is that students are forced to complete the process sooner when help is available. They receive more information about higher education costs sooner, which makes the college choice process more manageable. What we did not change in any meaningful way were the factors that lead to many students making last-minute decisions about whether or where to attend. For example, some students were unclear whether they would graduate high school, and some required additional summer coursework. Our goal with the project was not to suggest that every student should go to college but to keep the option open to as many students as possible.
