The Effectiveness of Networks and Independent Private Schools in Chile’s

National Voucher ProgramDante Contreras

Universidad de Chile

Gregory ElacquaPrinceton University

Felipe SalazarUniversidad Adolfo Ibanez

Prepared for the World Bank conference "Private-Public Partnership", June 7-8

Washington, DC

Persistent debate over the optimal scale of operations in schools

One view:

• Larger schooling operations offer educational services more effectively and efficiently than small independent schools (Whittle, 2000, Chubb, 2001)

– Economies of scale in education – By infusing competition and a business approach to education, schools will

succeed (or fail) like businesses

• School networks* promote sound institutional environments in member schools (McMeekin, 2003)

– Provide a sharing experience and facilitate the flow of information to network members (share information on best practices)

• A network can provide political benefits and credibility and legitimacy in a school’s community.

– Charter school networks more easily access loans and grants and raise funds in the community than independent charter schools (Wohlstetter and Smith, 2006)

*We define networks as schools that belong to a chain of schools that are operated by the same legal private voucher school “owner” (sostenedor).

Skeptics are concerned that…

• Large schooling operations will create hard to manage bureaucracies and foster diseconomies of scale due to associated problems of managing complex organizations.

• Also worry that large schooling operations: – Lower staff motivation b/c empower administrators far removed from the

classroom (Lawrence et al.,2002)– Lower student motivation and parent involvement

• they will experience isolation and anonymity in large organizations (Hill and Bonan, 1991)

– Promote more standardization and less innovation • School networks must establish a brand, which necessitates relative uniform

operations and services from site to site, which stifles innovation (Belfield and Levin, 2005)

• They claim that small autonomous schools can improve the quality of education by creating intimate learning communities with higher levels of trust, where teachers and parents are essential to school governance (Bryk and Schneider, 2002).

These assertions have sparked two distinct trends in school management

• District consolidation and funding for private school networks – In the US, district consolidation is one of the most significant

reforms in education governance and management in the 20th century (Tyack, 1974)

• Over 100,000 school districts have been eliminated since 1934 (90% decline)

– Growing number of private school networks (e.g. Edison Schools) and charter school partnerships

• Small schools initiative– Large schools reorganized into smaller autonomous schools

• The Bill and Melinda Gates Foundation has already invested over US$ 1 billion to divide large urban high schools in the United States.

Evidence on size of schooling operations

• Empirical evidence on optimal size of schooling operations is limited:– Clouded by methodological limitations (Andrews et

al., 2002)– There are so few educational systems that provide

public funding to private schools (OECD, 2003) and nonprofit status is usually required for private educational institutions (James, 1993).

– Research suffers from thin data (small-scale programs or case studies)

• Evidence on small scale programs is mixed (Gill et al., 2007; AIR & SRI, 2007)

This paper…• This paper compares achievement of 4th graders

in private voucher school franchises, independent private voucher schools, and public schools in Chile.

Why Chile?

• Chilean case public funding for private schools.• Chilean case nonprofit status is NOT required for

private educational institutions.• Chilean case LARGE-scale voucher program.

Research in Chile

• This is not the first paper to compare private and public school achievement:– Aggregate school level data (e.g., Bravo et al. 1999, Contreras, 1999,

Mizala and Romaguera 2000)– Student level data that attempts to control for selection bias (e.g. Anand

et al., 2006: Sapelli and Vial, 2002; McEwan, 2001).

• Most studies show a small private school advantage

• This paper differs from earlier work by examining the effect of size of schooling operations on student achievement.

Background on Chile• In 1981, Chile’s military government established a “textbook”

voucher scheme, by providing vouchers to any student wishing to attend a private school, and by directly tying the budgets of public schools to their enrollment.

• Reform sparked a massive redistribution of students across private and public schools.

• More than 1000 private voucher schools (most for-profit schools) entered the market and the enrollment rate increased from 16% in 1981 to 42% today, surpassing the 50% mark in many urban areas.

• Essential features of Chile’s national voucher program remain unchanged 26 years later.

Distribution of students across school type (1979-2003)

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

1979 1981 1986 1993 2003

Year

% t

ota

l pri

ma

ry a

nd

se

co

nd

ary

Public Private voucher Private non-voucher

Size of schooling operations in Chile: 68% of private voucher schools are independent schools

12%

4%2%2%

12%

68%

1 school

2 schools

3 schools

4 schools

5 schools

>5 schools

Total private voucher schools: 2,872

Total private voucher students : 914,439

Empirical strategy

• We use two-step procedure to control for school choice.

• We replicate previous estimates provided in literature (Journal of Human Resources, 2000; Education Economics, 2001).

• We are using same variables and empirical strategies and student level test score data.

• Goal: identify some variation (if any) in the documented private-public gap by school network size.

Empirical strategy

• Our model builds on previous work by McEwan (2001).• We hypothesize that student achievement can be

modeled in the following form:

• the test score A of ith student in jth school type is a function of independent variables that describe individual, family, school and community characteristics and an error term.

• Using the estimates of we can predict the achievement of a “typical” student in each school category.

ij ij j ijA X

Selection bias

• If independent variables account for student and peer demographics, then the above strategy yields unbiased results.

• More likely is that variables are either imperfectly measured or omitted from the regression.

• The average student attending private schools may be more likely to have certain attributes (i.e. they may value education) than the average student attending public schools (parental choice bias).

• Private schools also may select more qualified students than public schools (school choice bias).

• A simple comparison is unlikely to give unbiased estimates of impact of networks (or independent schools) on student achievement.

To correct this problem Lee (1983) has developed a two-step selection bias procedure for cases where choice is among several alternatives (parental choice bias).

In the first step the decision to choose a school type (network, independent, public) is estimated as a multinomial logit. The estimates from the multinomial logit are used to construct a selectivity term for each observation, which then becomes an independent variable in the achievement regression.

ij ij j ij j ijA X

There is still room for worry…

• We expect independent variables that influence achievement to be similar to those that influence school choice. So the estimates may still be biased if some unmeasured variable affect achievement.

• The key empirical problem in implementing a two stage model is distinguishing the network effect (or small independent school effect) from the effect of other variables that are not observed.

• One approach to this problem is to find an exogenous variable that affects the probability of attending a private voucher school network (or independent) school and that is not correlated with the error term in the outcomes equation.

• Instrument We hypothesize that the probability of choosing a given school type is affected by the number of schools per square kilometer in her municipality (McEwan, 2001; Contreras, 2002).

• It is assumed that the density of schools is not correlated with achievement (our results corroborate this assumption).

We acknowledge that…

• Much of the debate around differences between public and private schools has revolved around the statistical techniques that purport to control for student background characteristics and potential selection bias.

• Rather than develop a new strategy or use a different strategy (i.e. propensity score matching) to account for selection bias, we build on previous published work that uses the same student level data, which allows us to compare our findings.

The control variables

• Student: Mother, father schooling, hh income, gender, books in hh.

• Regional dummies.

• Peer effects: Average SES in classroom.

• School level: tuition, school size (number of students) and rural

Data

• National standardized test (SIMCE) 4th grade, 2002.

• 274, 863 students evaluated

• Test scores complemented with parental information.

• Dependent variables: Spanish and mathematics test scores were standardized to a mean of 0 and a standard deviation of 1.

Empirical results

Voucher 1 Voucher >1

franchise SPANISH Unadjusted difference 0.321 0.476 [0.005] [0.006] Difference adjusted for:

Individual SES 0.110 0.286 [0.028] [0.041] Individual SES/peer SES 0.051 0.200 [0.025] [0.039] Individual SES/peer SES/selectivity 0.057 0.284

[0.033] [0.064] Number of observations 47,754 34,685

Voucher 1 Voucher >1

franchise MATH Unadjusted difference 0.296 0.440 [0.005] [0.006] Difference adjusted for:

Individual SES 0.063 0.296 [0.027] [0.043] Individual SES/peer SES 0.008 0.220 [0.025] [0.041] Individual SES/peer SES/selectivity 0.041 0.315

[0.033] [0.069] Number of observations 47,610 34,722

Difference in test scores between private voucher and public schools after controlling for SES, peer and selection bias: positive network effect and no statistically significant difference between public and independent voucher schools.

Voucher 1

Voucher 2 franchise

Voucher 3 franchise

Voucher 4 franchise

Voucher 5 franchise

Voucher >5 franchise

SPANISH Unadjusted difference 0.321 0.355 0.538 0.615 0.496 0.531 [0.005] [0.009] [0.013] [0.019] [0.019] [0.009] Difference adjusted for: Individual SES 0.110 0.216 0.246 0.368 0.420 0.287 [0.028] [0.067] [0.071] [0.086] [0.125] [0.059] Individual SES/peer SES 0.051 0.165 0.149 0.235 0.189 0.204 [0.025] [0.062] [0.065] [0.067] [0.138] [0.059] Individual SES/peer SES/selectivity 0.050 0.186 0.501 0.206 0.831 0.257 [0.032] [0.140] [0.130] [0.104] [0.167] [0.104] Number of observations 47,556 11,981 5,548 2,517 2,341 12,298 MATH Unadjusted difference 0.296 0.351 0.456 0.581 0.466 0.483 [0.005] [0.009] [0.013] [0.019] [0.019] [0.009] Difference adjusted for: Individual SES 0.063 0.222 0.214 0.372 0.432 0.312 [0.027] [0.068] [0.078] [0.108] [0.140] [0.065] Individual SES/peer SES 0.008 0.176 0.138 0.250 0.157 0.234 [0.025] [0.061] [0.089] [0.081] [0.164] [0.067] Individual SES/peer SES/selectivity 0.036 0.225 0.671 0.232 0.751 0.378 [0.032] [0.117] [0.180] [0.152] [0.225] [0.127] Number of observations 47,612 11,994 5,550 2,521 2,346 12,311 Source: Ministry of Education and authors’ calculations Note: Standard errors are in parentheses

Significant network effect for schools that belong to a network of 3 or more schools (10-40 additional points in SIMCE).

Are the magnitudes of these effects noteworthy?

• Previous work in Chile (McEwan, 2001) and the United States (Neal, 1994) find a Catholic school effect of about .09 of a standard deviation.

• We used the same empirical strategy, the same control variables, and the same student level data, and found that private voucher schools that belong to a network, on average, have quite substantial effect sizes, nearly one-half of a standard deviation (almost 25 points).

To probe these findings further…

• We need to make sure we are not confounding the effect of attending a private network school with the effect of a Catholic school.

• Previous research in Chile and the United States has demonstrated that Catholic schools, all else equal, outperform public schools and other types of private schools.

• Our findings do not change substantially after controlling for whether or not school owner is Catholic

Summary

Once student and peer attributes and selectivity are controlled for we find:

• Private schools that belong to a network have a substantial advantage over public schools.

• Schools that belong to a network with three or more schools outperform smaller networks and independent schools.

• No statistically significant difference in achievement between public schools and private independent voucher schools.

Some possible explanations for the positive network effect

• Substantial benefits of scale of educational professionals, bulk purchases, and the reduced costs of implementation of innovations in curriculum (e.g. Chubb, 2001).

• Larger networks may be more likely to benefit from access to credit and private investment than independent schools (Elacqua, 2007)

• Being embedded in a larger communal organization reduces agency problems and facilitates transactions between staff, parents and teachers (McMeekin, 2003) and influences the development of professional school communities (Smith & Wohlstetter, 2001)

However, before holding these results up as evidence of a network effect…

• We need additional information on the factors that may influence a private school owner to establish a network that may influence outcomes.– High achieving schools may be more likely to establish networks

and attract more students than low quality schools.– In a competitive environment, low quality schools may be unable

to attract students and resources needed to expand operations.– Networks may also require superior technical skills to manage

than small independent schools.

• Such causal effects is a topic for future research. We need more information.

From a policy perspective…

• More information is needed on the incentives that affect a school owner’s decisions to establish a network.

• How profitable are private voucher school networks? Is the education industry too “risky” for entrepreneurs?

• After 26 years of vouchers, still a cottage industry – 2/3 of private schools do not belong to a network

• Data on the characteristics of school owners would improve our understanding of the complex decisions involved in establishing a network.