Dr Iain ButterworthVisiting Senior FellowFaculty of Health and Behavioural SciencesSchool of Health and Social DevelopmentDeakin University

Healthy Cities Evaluation: Healthy Cities Evaluation: Tracking Empowering Efforts Tracking Empowering Efforts

and Outcomes Across the and Outcomes Across the Social SystemSocial System

http://www.health.vic.gov.au/localgov/

Uses for EvaluationUses for Evaluation

• Rendering judgments – did this program attain its goals?

• Facilitating improvements – what are the program's strengths and weaknesses?

• Generating knowledge – enlightening program staff and the wider public.

(Patton, 1997; http://www.si.umich.edu/Community/connections/archives/evaluationprep.html)

Evaluation must beEvaluation must be

• Practical

• Cost-effective

• Ethical

• Accurate

(Patton, 2002, www.wmich.edu/evalctr/checklists/ufechecklist.htm)

Characteristics ofCharacteristics ofHealthy Cities ProjectsHealthy Cities Projects

1. Commitment to health

2. Political decision-making

3. Intersectoral action

4. Community participation

5. Innovation

6. Healthy public policy (Duhl & Sanchez, 1999)

11 Key Parameters for Healthy 11 Key Parameters for Healthy Cities, Communities and TownsCities, Communities and Towns

1. A clean, safe, high-quality environment

2. A stable, sustainable ecosystem

3. A strong, supportive community.

4. A high degree of public participation in and control over the decisions affecting life, health and wellbeing.

((Hancock & Duhl, 1988)Hancock & Duhl, 1988)

5. The meeting of basic needs (food, water, shelter, income, safety, work)

6. Access to a wide variety of experiences and resources

7. A diverse, vital and innovative economy.

8. Encouragement of connections with cultural and biological heritage

11 key parameters…11 key parameters…

9. A city form (design) that enhances parameters 1-8

10.An optimum level of appropriate public health and sick care services accessible to all.

11.High health status (both high positive health status and low disease status).

11 key parameters…11 key parameters…

Change across the social systemChange across the social system

IndicatorsIndicators

• Variables with characteristics of quality, quantity and time used to measure, directly or indirectly, changes in a health and health-related situation and to appreciate the progress made in addressing it (WHO, 2002)

• Act as a tool for health promotion by raising public awareness of what is going well and what is threatening a community’s well-being (Conner et al, 2003)

Revised 32 Baseline Healthy Cities Revised 32 Baseline Healthy Cities IndicatorsIndicators (Doyle et al, 1999; WHO, 1998) (Doyle et al, 1999; WHO, 1998)

A Health indicatorsA1 Mortality: all causes A2 Cause of death A3 Low Birth weight

B Health service indicatorsB1 Existence of a city health education programme B2 Percentage of children fully immunizedB3 Number of inhabitants per practicing primary health care practitioner B4 Number of inhabitants per nurse B5 Percentage of population covered by health insurance B6 Availability of primary health care services in foreign languages B7 Number of health related questions examined by the city council every year

Revised Baseline Indicators…Revised Baseline Indicators…

C Environmental indicatorsC1 Atmospheric pollution C2 Water quality C3 Percentage of water pollutants removed from total sewage produced C4 Household waste collection quality index C5 Household waste treatment quality index C6 Relative surface area of green spaces in the city C7 Public access to green space C8 Derelict industrial sites C9 Sport and leisure C10 Pedestrian streets C11 Cycling in city C12 Public transport C13 Public transport network cover C14 Living space

Revised Baseline Indicators…Revised Baseline Indicators…

D Socio economic indicatorsD1 Percentage of population living in substandard accommodation D2 Estimated number of homeless people D3 Unemployment rate D4 Percentage of people earning less than the mean per capita income D5 Percentage of child care places for pre-school children D6 Percentage of all live births to mothers > 20; 20-34; 35+ D7 Abortion rate in relation to total number of live births D8 Percentage of disabled persons employed

Indicators must be:Indicators must be: Interesting Relevant Easily understood Accessible Fully supported by key stakeholders Directly applicable to Policy Developed using collaborative, iterative process Cost effective Useful

(Innes & Booher, 1999)

Website: www.civicpartnerships.org

Social ecology framework to assess Social ecology framework to assess changes in community capacity changes in community capacity

i. Changes in individuals

ii. Changes in civic participation

iii. Organizational development

iv. Inter-organizational activity

v. Community level changes

(Kegler, Norton & Aronson (2003)

Dimensions of Dimensions of community capacitycommunity capacity

• Civic participation• Mechanisms for community input• Mechanisms for distribution of community power• Skills and access to resources• Sense of community and social capital/trust• Social and inter-organizational networks• Community values and history• Capacity for reflection and learning

Psychological Sense of CommunityPsychological Sense of Community

A feeling that members have of belonging, a feeling that members matter to one another and to the group, and a shared faith that members’ needs will be met through their commitment to be together

((McMillan and Chavis, 1986)McMillan and Chavis, 1986)

Social CapitalSocial Capital

• Focus on networks between people that lead to cooperation, trust, social power and beneficial outcomes (Baum, 2000)

• Interpersonal, community, institutional, or societal levels (Perkins, Hughey & Speer, 2002)

Evaluation designEvaluation design

• Multiple case study with cross-case comparisons

• Data collection involved:– Review of program documents– Participant surveys in year 1 and year 3– In-depth interviews– Analysis by ‘type of community’

Individual-level changesIndividual-level changes

• More than 1,100 new civic leadership roles

• 1,500 people acted in these roles

• Urban participants reported gaining skills in advocating for policy change

Organizational Changes

• A broadened definition of health• Increased community input into decision-making• Expanded and new forms of collaboration• Adoption of shared vision• Expanded or restructured programs/services

aimed at increasing equity• Administrative policy changes designed to

enhance service delivery and equity

Community level changesCommunity level changes

• HC/C programs leveraged changes in other organizations

• Most programs reported at least one public (government) policy change arising from their efforts

• Seven of the 20 communities identified policy change as explicit goal area

Sense of Community:Sense of Community:

• Increased number of diverse participants the program

• Increased cooperation and communication in the community

• New gathering place and the opportunities the program provided for diverse people to interact and form relationships

Key categories of change to Key categories of change to physical environmentphysical environment

1. Neighbourhood and community beautification

2. Facilities construction, expansion and renovation

3. Public utilities and public safety

4. Parks and recreation facilities – construction and renovation

Psychological Predictors of Psychological Predictors of Social Capital:Social Capital:

• Place attachment• Community satisfaction• Communitarianism• Community confidence • Demographics• Collective efficacy• Participation in a neighbourhood association• Neighbouring behaviour

(Perkins & Long, 2002)

EmpowermentEmpowerment

• concerned with actual social influence and political power (Rappaport, 1987)

• long term process, many actions

• psychological, organizational, community

The acquisition of critical social insight, skills, and sense of competence needed to participate meaningfully in society (Kieffer, 1984)

Level of Analysis

Empowering Processes(focus on efforts)

Empowered Outcomes(results of efforts)

Individual • Participation in empoweringcommunity organizations

• Engaging in social critique• Reflecting on life experiences• Learning how social system works in

particular context• Learning how to obtain resources• Developing skills to manage

resources

• Intrapersonal empowerment: perceptionof having gained control over certainsituations

• Instrumental empowerment: gainingsociopolitical awareness

• Redefinition of self and personal history• Taking action to influence events:

gaining participatory competence

Group/organization

• Collaboration with mentors• Productive group meetings• Collective decision-making• Shared leadership• Peer support• Collective action• Participation in formal decision-

making processes• Action research approach

• Network development• Lobbying power• Secured resources• Enhanced organizational vision,

ideology, values, and culture• Improved organizational structure• Enhanced organizational effectiveness• Instrumental empowerment

Community • Development of coalitions• Stakeholders involved from across

levels of social system• Institutional support

• Development of more and bettercommunity resources

• Policy commitments; legislation• Improved collective lives• Substantive empowerment: ecologically

sustainable outcomes

(Balcazar et al., 1994; Kieffer, 1984; Perkins & Zimmerman, 1995; Rich et al., 1995; Zimmerman, 1995)

Some personal reflections…Some personal reflections…

• HC programs are conducted in a geopolitical space or locale

• HC programs are inherently about power and control:– Power of communities to determine and drive

their own health agenda– Political decision-making

• Empowerment framework can integrate place, community and social capital

Personal reflections…Personal reflections…

• Document community capacity building efforts and outcomes

• Use mixed methods– Qualitative and quantitative– Scales, indicators, interviews, focus groups

• Collect population measures: – Garrard, J., Hawe, P., & Graham, C. (1995). Acting locally to

promote health: An evaluation of the Victorian Healthy Localities Project. Melbourne, Australia: Municipal Association of Victoria.

Personal reflections…Personal reflections…

• Indicators need to be developed as part of participatory evaluation framework

• Use indicators to stimulate community thinking and action and to document process, as much as outcomes

• University collaborations invaluable