Why Subsidize Independent Schools? Estimating the Effect of a Unique Canadian Schooling Model on Educational Attainment

Round 1

Reviewer 1 Report

The topic addressed by the paper raises interest both in the academic community and among policy makers. The empirical methodology is robust and it convincingly shows that independent schools in Québec have better achievements than public schools.

Nevertheless, I am not sure that findings allow to understanding the mechanisms that explain why independent schools have such good performance. Understanding such mechanisms is crucial:

• to explain why literature findings are inconsistent, i.e. why in some countries independent schools achieve better educational attainments and in other countries they do not.
• from a policy point of view, as it can help to ameliorate the outcomes of all schools (public as well).

The lack of an adequate discussion of possible mechanisms undermines, in my view, the relevance of a well-conducted analysis.

What are the mechanisms candidate to explain findings?

The study considers in a proper way the role played by students’ family characteristics, i.e. the demand side of the “school market”. The offer side, i.e. the characteristics of the schools, is however rather neglected.

Authors discuss the teachers’ selection issue by affirming that teachers in public and independent schools are similar. Teachers quality is one of the possible mechanisms. I would find informative to have evidence about teachers’ quality in public and independent schools.

Other potential mechanisms that should be investigate or, at least, discussed are:

• Study time. What is the study time in public versus independent schools? Do pupils in independent schools spend more hours in classrooms? Do they have more support from their teachers or other staff? Do they benefit from after-school programs?
• Class size. Is the average class size similar in independent and public schools? How many students per teacher are in different types of schools?
• Peer effects. What about “peer” effects? The methodology correctly considers the non-random selection of students into independent schools but not the non-random selections of peers. If peers are “good”, teachers can go fast, complete the programs or even provide insights on some topics. In short, they can reinforce students’ competencies. On the contrary, if peers are “bad”, teachers have to catching them up, slowing explanation and shortening the program. It is well known, in education literature, the role of peers.  This point relates to the following one.
• School autonomy. Do schools follow the same national (provincial) teaching programmes? Or are they autonomous in selecting programmes, topics, etc., i.e. are they truly “comparable”?

All in all, my guess is that the question to be answered should be: in order to improve students’ performance, do governments have just to encourage the opening of new independent schools, or do they have to implement some specific features of independent schools to all schools?

I hope these suggestions may enhance the (already good) analysis.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

Dear authors,

The structure of the article is appropriate, and the text is well structured; and the literature cited is updated and in accordance with the subject under study.

I rate the article as excellent and the study have high scientific value, however, I ask the following:

• the students belong to different schools, to different classes and are accompanied by different teachers, therefore the (students') observations are not independent. How do you control this situation?
• Can you include for example the size of the class? According to some literature on the subject, class size has a significant impact on students' performance.

Kind regards,

Comments for author File: Comments.pdf

Author Response

Thank you very much for your positive appreciation of our work. Here are the answers to your queries, we hope they are satisfactory.

Your comments:

• the students belong to different schools, to different classes and are accompanied by different teachers, therefore the (students') observations are not independent. How do you control this situation?

 

Regarding your query concerning how we treat the dependence of observations because of students attending the same school or class. We follow Statistics Canada guidance for the computation of standard errors for our causal estimates using the bootstrap re-sampling method. According to Statistics Canada: “This technique is popular among surveys with a large number of strata and multiple primary sampling units (PSU) per stratum. Unlike the Jackknife method the bootstrap does not suffer from inconsistent estimates for population estimates such as percentiles.

The idea behind the bootstrap method is to select random sub-samples from the full sample in such a way that each of the sub-samples (or replicates) follows the same design as the full sample. The initial bootstrap weight is calculated by multiplying the initial sampling weight by a factor that accounts for the bootstrap sampling for those units selected in the bootstrap sample. For those units not selected in the bootstrap sample the bootstrap weight is equal to zero. The weights for units in each replicate are recalculated, following the same weighting steps used for the full sample…These are the final bootstrap weights which are used to calculate a population estimate for each replicate. The variance among the replicate estimates for a given characteristic is an estimate of the sampling variance of the full-sample population estimate.”

Because the design of the survey is based on choice of schools and choice of students within schools, sampling weights reflect this and the bootstrap replicate weights are also structured by the design of the survey.

 

Your comment: 

• Can you include for example the size of the class? According to some literature on the subject, class size has a significant impact on students' performance.

Answer:

Regarding the possible use of class size as a control variable, class size is a possible mediator of independent schooling effects on attainment therefore our strategy was to avoid these as much as possible in our analysis, but the literature on the impact of class size rarely shows a strong impact on attainment see https://wol.iza.org/uploads/articles/190/pdfs/class-size-does-it-matter-for-student-achievement.pdf.

Round 2

Reviewer 1 Report

In the current version authors offer a discussion of the potential mechanisms which can explain the outcomes of the independent schools in Canada. Therefore,  the analysis is complete and I agree with the publication of the paper in this version.

This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.