Choice Experiments and Environmental Benefits Transfer

Nick Hanley and Sergio Colombo

The issue

• “Benefits transfer” is becoming increasingly important in environmental policy making and implementation

• Best example in Europe at present: “disproportionate cost” assessments under Water Framework Directive

• But lots of other examples: re-designing agri-environmental policy, managing forests to maximise social benefits, appraising landscape impacts of transport projects, benefits of improving air quality in cities, noise externalities from transport schemes….

• It is often too expensive and time-consuming to do original environmental valuation studies, so need benefits transfer

• BUT – errors, when we test them, often seem “large” and “unpredictable”

• How to improve things? What seems to drive the size of transfer errors?

What is “Benefits transfer”?

• Have valuation data on some “original” site eg water quality improvements on the River A. Known as the study site.

• Want to apply this to some other context eg similar water quality improvement on River B (known as the policy site); but have no time/money to do a valuation study

• So transfer either the mean value from A to B, perhaps adjusted to account for environmental or social differences; or transfer the “value function” from A and use it to predict values on B

• Transfer error is the difference between the transferred value for B, and the value you would have got from an original study, expressed as a %.

What determines the size of transfer error?

• How similar are study site (s) and policy site(s) in terms of environmental characteristics and characteristics of benefitting population

• Less obviously, how we estimate the “source” choice model, and

• what procedure is adopted for the transfer itself seems to matter

The effects of how the “source” choice model is estimated

• Most early choice experiments (CE) assumed preferences were the same for all in the sample (use of conditional logit models) or imposed “exogenous” splits on groups of people eg rural versus urban

• However, we expect that people will vary in how much they value the different attributes of an environmental good

• Several ways have emerged of representing this preference heterogeneity in the literature

• Two most common in CE are Random Parameters (“continuous mixing) models and Latent Class (“finite mixing”). A third approach is the covariance heterogeneity models of Bhat and Swait/Adamowicz.

Does allowing for preference heterogeneity make a difference to benefits transfer

errors?

• Study by Colombo et al on costs of soil erosion in Southern Spain (AJAE, 2007).

• Ran same choice experiment in two neighbouring catchments, The Genil and Guadajoz. Off-site impacts of soil erosion currently severe in both. Management actions to reduce erosion were the goods being valued.

• Attributes: levels of landscape desertification, impacts on water quality, impacts on biodiversity, area of project, jobs in agriculture.

• Compared transfer errors between two catchments for implicit prices and compensating surplus using (i) standard conditional logit model (ii) random parameters logit model.

• Results: using RPL and including socio-economic characteristics improved model fit a lot.

• Across 27 policy scenarios, we reject the transferability of compensating surplus estimates at the 95% level

• BUT: average transfer error is much smaller with RPL than with CL model: 66% versus 154%

• Conclusion: yes, it does make a difference

• But why?

Estimating the source choice model (2)

• Another feature of early CE studies, using both CL and RPL, is that we essentially assume peoples preferences for the different attributes of an environmental good are independent of each other

• But in fact, there might be quite a bit of correlation between my preferences for say ecological quality and for the aesthetic appearance of a river

• Allowing for all possible correlations between preferences makes the estimation much more complicated, but does it make a difference to benefits transfer?

Evidence:

• Hanley et al study of benefits transfer for 2 neighbouring Scottish rivers, both subject to low flow and nutrient enrichment problems (ERAE, 2006)

• Choice experiment study on “locals” in each case• Compared transfer errors between rivers with and without allowing

for correlation• Result: makes a difference to our benefit transfer tests of equality,

especially where we use an equivalence test rather than the usual Poe et al test (correlated coeffs. version makes it more likely we reject the transferability of compensating surplus estimates between rivers).

• Although makes no difference to transferability of implicit prices• But again, why?

How we undertake the transfer process: Pooled versus individual site transfers

• Often we have more than one study available on which to base benefit transfers – and often feel that more is better.

• But is it? And does it matter which sites we “pool” to produce our benefit transfers?

• We investigated these issues in a CE of landscape features in upland England.

• Colombo and Hanley, Land Economics, forthcoming.

landscape attributes

• heather moorland and bog*;

• improved grassland;• rough grassland*;• hay meadows;• bracken;• gorse;• arable & set aside

land;

• broadleaf and mixed woodland*;

• coniferous woodland;• field boundaries*;• cultural heritage*;

Survey Design

• 300 households randomly sampled in each of six regions which have “Severely Disadvantaged Areas” (SDAs) in them –North West, North East, Yorks and Humberside, West Midlands, East Midlands, South West.

The questionnaire contained 3 sections:• Attitudes and opinions on the countryside• Choice Experiment tasks• Data on visits to SDAs, and on socio-economic

characteristics

Policy Option Current Policy

Policy Option A

Policy Option B

Change in area of Heather Moorland and Bog

A loss of 2% (-2%)

A gain of 5% (+5%)

A loss of 2% (-2%)

Change in area of Rough Grassland

A loss of 10% (-10%)

A gain of 10% (+10%)

A loss of 10% (-10%)

Change in area of Mixed and Broadleaf Woodlands

A gain of 3% (+3%)

A gain of 20% (+20%)

A gain of 10% (+10%)

Condition of field boundaries

For every 1km, 100 m is restored

For every 1km, 200 m is restored

For every 1km, 50 m is restored

Change in farm building and traditional farm

practices Rapid decline

Much better conservation

No change

Increase in tax payments by your household each

year £0 £40 £17

Which do you like best?

There is much variation in the factors that influence the choices respondents made across the 4 SDA regions.

Heather moorland and bog and “much better conservation” of cultural heritage are shown to be significant factors in respondents’ choices in all regions.

Changes in broadleaved and mixed woodland significantly affected choice in two of the four SDA regions.

On the other hand, changes in rough grassland and “no change” in cultural heritage (relative to rapid decline under the status quo) are only significant factors in choices in one region,

Whilst changes in field boundaries have no significant effects on choice.

Benefits transfer tests

• These were to transfer compensating surplus estimates for three different future policy options, which would impact on these upland features, across “sites”, where “sites” are Severely Disadvantaged Areas in England

• Focus on North West, Yorks and Humber, West Midlands and South West (ie 4 sites)

• Results: transfer error depends on which site or sites one picks as the “study site”.

For instance, imagine the policy maker had chosen the South West as the study site, and transferred values to the other three regions based on the model estimated using data from the South West. For policy scenario 1, this would give a transfer error of 258% for the North West,63% for Yorkshire/Humberside and 36% for the West Midlands.

If instead the policy maker had chosen the West Midlands as the study site from where original data was collected, the transfer errors for the same scenario for other three regions (North West, Yorkshire/Humberside and South West) would be 163%, 19% and 27% respectively.

But how could we tell ex ante which was the “best” study site?

Pooled versus single site transfers

• transfer errors are not always smaller by using a pooled approach rather than a single site approach.

more information does not always improve benefits transfer error

We also compared using just a few sites (even one site) selected on the basis on “site similarity” measures – eg based on:– population characteristics, or– landscape features or– proximity.

with the “maximum information” approach of pooling data on all sites and using this to predict values at a new site

• Results: errors from pooled models depend on which sites you pool; whilst choosing sites on basis of similarity indicators does not always reduce transfer errors.

`

• In fact, when site similarity is taken into account, the expected transfer error from using pooled versus single site transfers is about equal!

• We find that a 10% increase in survey costs reduces transfer error by 2.6%-4.2%

• Conclusion: again, more information is not always better, especially when it costs!

Ongoing work…

• Joint with Garth Holloway (Reading)• Using Bayesian approaches to see if smaller transfer

errors result, based on both our Spanish soil erosion and English upland landscapes data sets, and on simulated data.

• One idea: using “pilot study” sample sizes from policy site to update estimates of choice model from study site.

• Early indications are that this can reduce transfer errors by a lot

• Paper on methodology being presented to Spanish/Portuguese environmental economics association conference in June.

On going work (2)

• Using distance decay functions to help with aggregation process

• Find way of estimating what is the “relevant population”• Earlier CV study on river flows: produced “convenient”

relationship of distance to WTP for both users and non-users, continuously declining with distance (Hanley et al, JEM, 2003). Bateman et al find similar functional form.

• Tried similar thing in current CE study on Manchester Ship Canal.

• Distance-decay relationship estimated as quadratic in a choice model

• Most of motivation for WTP is non-use.

• Can use to produce estimates of aggregate benefits in each distance band, but we don’t know how far have to go away from canal for wtp 0.

Conclusions: moving forwards on benefits transfer

• Still much to learn• Start from a good representation of underlying

preferences – ie make the “original” study a good one, and one which allows “important” socio-economic drivers to be included.

• More information is not always desirable• Develop/test indicators of site similarity• “quick and dirty” Bayesian updating approach promising

• How close is “close enough”? Depends on context.