Experiments in Measuring Sustainability - The Environmental Sustainability Index and its Critics

Marc LevyCIESIN

marc.levy@ciesin.columbia.eduhttp://sedac.ciesin.columbia.edu/es/esi/

CIESIN involvement with sustainability indicators

• 1999-2005– Environmental Sustainability Index

• Collection of national-level indicators suitable for comparison and aggregation

• 2005-2006– Environmental Performance Index

• Collection of national-level “report cards” measuring proximity to policy “targets”

– Proposal to U.S. Millennium Challenge Account currently in public review (http://www.mca.gov/countries/selection/NRS_indicator.shtml)

• 2002-2006– Collection of integrated well-being / environment indicators to

support research into systemic interactions• E.g. Millennium Ecosystem Assessment

Sustainability Indicators in Context

Understanding

Goal-Setting

Implementation

Eva

luat

ion

and

Lear

ning

Engagem

ent and D

eliberation

Useful Indicators can improve ability toUseful Indicators can improve ability to- Describe - Describe problems accurately and salientlyproblems accurately and saliently- Diagnose- Diagnose the causes of these problems the causes of these problems- Design- Design solutions commensurate with description and diagnosis solutions commensurate with description and diagnosis - Drive - Drive action with ongoing monitoring and evaluationaction with ongoing monitoring and evaluation

Criticism

• Multi-dimensionality plus aggregation = confusion

• Aggregates not grounded in theory; not subject to testing

• Weights are ultimately arbitrary

Response• There is a demand for aggregated

numbers

• Although aggregation can be misused, it can be useful

• Transparency can temper arbitrariness

The ESI gives strong weight to social and institutional capacity

measures• Social and Institutional Capacity one of

five core components of the ESI– 4 of the ESI’s 21 indicators are capacity

measures• Governance• Eco-efficiency• Private Sector Responsiveness• Science and Technology

– 24 variables used to quantify these indicators

The critique

• “Rewards” wealthy countries

• Capacity measures aren’t environmental, so they cloud the picture of environmental sustainability

• Some of attributes of high capacity are linked to patterns of high environmental stress (e.g. resource consumption)– might send wrong signal

Response

• Stick with it because it matters

• Erect clear boundaries separating governance from drivers and impacts

The ESI combines things that are within governments near-term control and those that are not

• Exposure to environmental natural hazards

• Endangered species• Anthropogenic land conversion• Projected population growth

• Things that happened long in the past aren’t relevant for current planning

• Things that can’t be controlled aren’t relevant

Criticism: This confuses whatever signal the ESI wants to send about

performance

Response, I

• Given what ESI was trying to quantify, it makes sense to include both kinds of metrics– ESI is meant to measure ability to maintain

favorable environmental conditions long into the future

– That is a function of the cumulative, interacting effects of exogenous conditions, behaviors undertaken in the past, and behaviors undertaken in the future

Response, II

• A Pilot Environmental Performance Index– Focuses only on measures

subject to policy intervention– Metrics are benchmarked in

terms of proximity to target

• Natural Resource Management Indicator– Proposed for use by

Millennium Challenge Account• Unweighted average of

– Access to water– Access to sanitation– Child mortality (age 1-4)– Achievement of 10%

protection target, by biome

D P S R I c o l l e c t i

o n s ( E U ,

O E C D ,

U N E P )

E E A R e p o r t

C a r d s

E S I

Narrow

E c o l o g i ca l

F o o t p r i n

t

G r e e n A c c o u n t

i n g

Absent C S D E V I

Absent Implicit Explicit

Tre

atm

en

t o

f s

ys

tem

ic

inte

rco

nn

ec

tio

ns

Broad / Explicit

Broad / Implicit

Treatment of Goals

Zone of Silence

Clear Sustainability Targets Remain Elusive

• Human-oriented indicators tend to be linked to clear targets– Child Mortality– Drinking Water– Sanitation– Urban Particulates– BIG EXCEPTION: Indoor Air Pollution

• Ecosystem-oriented targets hard to find– Regional ozone– Nitrogen loading– Water consumption– Wilderness Protection– Overfishing

These are problems that manifest themselves over complicated transnational,multi-scale, coupled-system dynamics

Measurement Infrastructure is not Adequate

• Of the 16 indicators included in EPI, only 9 are updated on a regular basis– The indicators measured regularly are dominated by

human-focused indicators

• This reinforces the current policy stalemate– Hard to set goals when metrics aren’t available– Hard to mobilize support for measurement in the

absence of policy goals– MDGs help reinvigorate many socioeconomic

measurement efforts – did not have same effect on the environment