kwaw s. andam international food policy research institute

INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE

What are the Effects of Land Use Restrictions on Local Communities?

Evidence from an Impact Evaluation of Costa Rica’s Protected Areas

Kwaw S. AndamInternational Food Policy Research Institute

Perspectives on Impact Evaluation: Approaches to Assessing Development Effectiveness

31 March - 2 April 2009, Cairo, Egypt

INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE

Acknowledgements

Co-authors: Paul Ferraro (Georgia State Univ.) and Margaret Holland (Univ. of Wisconsin-Madison)

Evaluation Office, Global Environment Facility (GEF)

INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE

Research Question

How different would socioeconomic outcomes have been in the absence of protected areas?

INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE

Policy Context

Protected Areas:

• Most widely used conservation tool

• Role in climate change policy?

• More planned for future

INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE

Motivation

Strong debate: how do protected

areas affect local people?

Most studies focus on

environmental impacts only

INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE

The Evaluation Challenge

Protected Areas placed selectively

Selection biasSelection biasLack of common supportLack of common support

Millennium Ecosystem Assessment (2005):

“Many protected areas were specifically chosen

because they were not suitable for human use.”

Empirical evidence: e.g. Costa Rica (Powell 2000; Sanchez-Azofeifa);

Nepal (Hunter & Yonzon 1993) ; Australia (Pressey 1995) ; United States

(Scott et al. 2001) ;

INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE

Therefore, an evaluation must…

1) Objectively measure indicators of human welfare

2) Measure indicators before and after establishment of protected area

3) Measure indicators in both “treated” and “control” areas

4) Measure baseline characteristics that affect both location of protected areas and how indicators change over time

INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE

Study site – Costa Rica

INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE

Costa Rica’s Protected Areas

Protected before 1980

Protected after 1980

Forest cover 1960

INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE

Approach

Estimate impact of protected areas established before 1980 on changes in census tract-level socioeconomic indicators between 1973 and 2000

INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE

Key question

How different would socioeconomic

outcomes have been in the

absence of protected areas?

INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE

Data

Quantitative indicators of change in socioeconomic outcomes (infrastructure, assets, poverty indices)

Control variables including land use productivity, forest cover (1960), accessibility (distance to markets, ‘road-less volume’), baseline (1973) indicator

Measures near time of and after establishment of protected areas

INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE

Matching Methods

Select control communities similar to communities near protected areas (treated) in terms of pre-protection characteristics

Key assumption: without protection (and conditional on control variables), control and treated communities would, on average, have similar socioeconomic outcomes in 2000

Treatment: at least 20% protected before 1980

• >>> any remaining differences in outcomes are due to protection

INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE

Variable Sample Mean ValueProtected Segments

Mean Value Control

Segments*

Diff Mean Value

Avg. Raw eQQ‡

Forest Area in 1960 (km2) UnmatchedMatched

15.81615.816

0.8338.618

14.9837.198

14.8976.543

High Productivity Land ◘ (km2)

UnmatchedMatched

0.8850.885

0.5401.012

0.345-0.127

0.2990.191

Medium Productivity Land (km2)

UnmatchedMatched

1.9221.922

0.4021.671

1.5200.251

1.4670.263

Medium-Low Productivity Land (km2)

UnmatchedMatched

4.3754.375

0.6503.584

3.7250.791

3.5940.719

Roadless Volume (km3) UnmatchedMatched

319.040319.040

21.322172.510

297.718146.530

291.960133.220

Distance to City (km) UnmatchedMatched

53.83653.836

35.34553.219

18.4910.617

18.4354.476

Poverty Index in 1973 UnmatchedMatched

1152411524

8574.20012016

2949.800-492.000

2950.100964.230

Percent of houses in bad condition in 1973

UnmatchedMatched

18.34318.343

15.01719.573

3.326-1.230

3.5041.313

Percent of houses in slums in 1973

UnmatchedMatched

1.4541.454

1.9421.486

-0.488-0.032

0.6030.150

Percent of houses without telephones in 1973

UnmatchedMatched

99.30499.304

93.60799.468

5.697-0.164

5.8100.144

Percent of houses without electricity in 1973

UnmatchedMatched

4.3654.365

3.0153.832

1.3500.533

1.2560.544

Percent of houses without access to water supply in 1973

UnmatchedMatched

39.46239.462

16.32140.458

23.141-0.996

23.0641.248

◘ Low productivity land is the omitted category.* Weighted means for matched controls.‡ Mean difference in the empirical quantile-quantile plot of treatment and control groups on the scale in which the covariate is measured.

INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE

Results

1 2 3 4Outcome % Houses in

bad conditionPoverty Index

% Houses in slums

% Houses without telephones

Matching Estimators (Effect of protection on change in outcome 1973-2000)Covariate matching – Mahalanobis

-9.3***(2.5)

-987.6***(248.9)

-3.4***(0.8)

1.2(1.5)

Covariate matching – Mahalanobis with calipers

-3.0**(1.2)

-500.2***(134.6)

-1.3***(0.5)

1.3(1.3)

Replicating Conventional Methods (Effect of protection on outcome in 2000)

Difference in meansN treated(N available controls)

399(16002)

399(16002)

399(16002)

399(16002)

*** significant at .01; ** p<0.05; * p<0.10

INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE

Additional Results

1 2 3

Outcome Unsatisfied basic needs (3 or 4)

% Houses without electricity

% Houses without water supply

Matching Estimators (Effect of protection on outcome in 2000)

Covariate matching – Mahalanobis

-14.1***(3.0)

-20.7***(4.6)

-10.7**(4.4)

Covariate matching – Mahalanobis with calipers

-8.5***(1.3)

-9.4***(1.5)

1.7(2.0)

Replicating Conventional Methods (Effect of protection on outcome in 2000)

Difference in means 8.7*** 16.8*** 31.1***

N treated(N available controls)

399(16002)

399(16002)

399(16002)

*** significant at .01; ** p<0.05; * p<0.10

INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE

Caveats

Average effects only (adverse effects on subgroups may still be present)

Costa Rica may be very different from other nations

Indicators may not capture other aspects of poverty

INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE

Ongoing work (1)

Census segments change over time from 1973 to 2000

Need to account for changes in unit of analysis over time

INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE

1973: 4,694 segments

INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE

2000: 17,264 segments

INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE

Ongoing work (1)

Use simple areal weighting to reconcile 1973 segments with disaggregated segments in 2000, and analyze at level of 1973 segments

Findings confirm analysis with disaggregated segments

INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE

Ongoing work (2)

Spillover effects: do effects of protection spillover onto nearby ‘unprotected’ communities?

Findings: None to small positive spillover effects

INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE

Ongoing work (3)

Test sensitivity of results to hidden bias (unobservable characteristics that affect likelihood of protection and socioeconomic outcomes)

Test using Rosenbaum bounds tests So far, mixed results

INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE

Summary

No evidence that protected areas had negative effects

Protected areas may have had positive effects

But conventional methods (erroneously) imply the opposite

INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE

Discussion

Why differences in estimates?• Simply comparing post-protection outcomes

does not account for pre-existing differences correlated with poverty indicators

How does protection have positive effects?• Ecotourism• Protected area infrastructure• Initiatives to ease deforestation pressures• Ecosystem services?

INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE

Thank you!

Kwaw S. AndamInternational Food Policy Research Institute

Addis Ababa, EthiopiaEmail: k.andam@cgiar.orgPhone: ++251.617.2507

www.ifpri.org

INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE

Appendix

More comparisons with conventional estimates

INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE

Replicating conventional methods1 2 3 4

Outcome % Houses in bad condition

Poverty Index % Houses in slums

% Houses without telephones

Matching Estimators (Effect of protection on change in outcome 1973-2000)

Covariate matching – Mahalanobis

-9.3***(2.5)

-987.6***(248.9)

-3.4***(0.8)

1.2(1.5)

Covariate matching – Mahalanobis with calipers

-3.0**(1.2)

-500.2***(134.6)

-1.3***(0.5)

1.3(1.3)

Replicating Conventional Methods (Effect of protection on change in outcome 1973-2000)

Difference in means 2.8*** 129.1*** 1.2** 23.4***

OLS2.4***(0.8)

709.6***(114.8)

0.6*(0.3)

11.3***(1.5)

N treated(N available controls)

399(16002)

399(16002)

399(16002)

399(16002)

*** significant at .01; ** p<0.05; * p<0.10