1

Qualitative Indicators TrainingDanish Child and Youth Network

Dr Leslie Groves

Social Anthropologist

2

The Workshop

1. Introduction to Indicators

2. How to Identify the ‘Right’ indicators

3. Collation and dissemination

4. Theory into Practice:Thematic Case Study work

5. Peer Panel review

6. Commitments and Close

3

1. Introduction to Indicators

4

Introduction to Indicators

What are they? Why are they used? Who uses them? When are they used? What do they measure?

5

What are they? ‘An indicator indicates’: shows you

something or points in a particular direction

Tool to understand and follow change Requires elements of comparison:

Time, place and proportion Quantitative and/or Qualitative Subjective

6

Why are they used? Baseline indicators: Establish current

situation and provide a reference for future work

Monitoring indicators: Show changes over time & check effectiveness of interventions and altered circumstances

Early warning indicators: Provide danger signals of deteriorating conditions in situations of sudden or unexpected change.

7

Who uses/ collects them?

Who decides the indicator and who does the research has important implications.

Different groups have different interests – donors, project implementers, target group, different groups within the target group.

Be careful of ‘Blind spots’ e.g gender, disability etc.

Collation not neutral process: gender, age, background, attitudes, behaviours etc.

8

When are they used?

Throughout project/programme/policy cycle

9

What do they measure? Input indicators: measure resource allocation:

number of health workers trained, number of buildings used as clinics etc. Usually at/ close to the start of project.

Performance/Process indicators: measure activities to track progress towards the intended results: regular meetings of women and health care workers, number of visits to mobile clinic, by sex of mother and child, views of activities of health workers and clinic, of parents

Progress/Outcome indicators: measure long-term results and whether contributed to goal: e.g reduced child, and maternal mortality and morbidity rates as compared to the national average within five years.

Impact indicators: measure medium and long term impacts of project. Wider than outcomes.

10

Where do you get the information from ?Quantitative: Population census, Household survey, School

surveys, Local govt files

Qualitative: Focus group discussions, interviews PRA: matrix ranking, transect walks,

mapping… Observation

11

2. How to Identify the ‘Right’ indicators

12

2. The ‘right’ indicators should be ‘Smart’: S pecific

M easurable

A chievable

R ealistic

T ime bound

Plus

Comparable: Capable of being used accurately in more than one situation Disaggregated: non- discrimination Cost effective: various sources Linked: to objectives etc. Participatory: Involving all stakeholders

13

…Then… ask yourself…

What assumptions do you have about behaviour change e.g education leads to behaviour change. Are there any other ways that you may not be thinking about? Be creative or you may miss important impact.

 Who are you leaving out? Have you done a good stakeholder analysis? Diversity issues, most marginalized groups.

Do you have a sufficient understanding of causes/barriers to behavioural change?

How are you building ownership? Particularly imp when looking at behavioural change.

14

Today:

Qualitative Indicators that highlight changes in:

Awareness Attitudes Behaviours Practices Norms and Values

15

Qualitative Indicators …

Ask ‘Why’, as opposed to how many Depth, as opposed to breadth Subjective, as opposed to objective Exploratory, as opposed to definitive Provide insights, as opposed to level Interpret, as opposed to ‘scientific’

16

Behavioural Change Indicators: Change For Who?

Individual level: school teachers, young sexually active persons, children, etc.

Household/family level: parent-child-sibling relationships, discussion of sexual health for ex.

Community level: teachers associations, youth clubs, health centres. Groups that discuss exclusive breast feeding etc.

Policy and Programming level: organisational, govt policies and actions passed

17

SC UK Common dimensions of changeCentral America example

1. Changes in the lives of children and young peopleWhich rights are being better fulfilled? Which rights are no longer being violated?

2. Changes in policies and practice affecting children & young people’s rightsDuty bearers are more accountable for the fulfilment, protection and respect of children’s

and young people’s rights. Policies are developed and implemented and the attitudes of duty bearers take into account the best interests and rights of the child.

3. Changes in children’s & young people’s participation and active citizenshipChildren and young people claim their rights or are supported to do so. Spaces and

opportunities exist which allow participation and the exercise of citizenship by children’s groups and others working for the fulfilment of child rights.

4. Changes in equity and non-discrimination of children & young peopleIn policies, programmes, services and communities, are the most marginalised children

reached?

5. Changes in civil society and communities capacity to support children’s rightsDo networks, coalitions and/or movements add value to the work of their participants? Do

they mobilise greater forces for change in children and young people’s lives?

18

1. Changes in the lives of target group Physical health: nutrition, development, substance

abuse, disturbed sleep

Mental health: perceptions of well-being, confidence, self esteem, antisocial behaviour, insecurity/ inhibition, self denigration-self worth, Apathy- autonomy

Improved relationships: school, families, communities, positive adult-peer relations, social integration

Improved access to basic services, work patterns,

Knowledge on rights

19

2. Changes in policies and practice

Organisation level: Staff motivation, capacity, awareness, changing work practice such as participatory work increased. Improved partnerships, linkages

Wider level: National/local policies (PRSPs, NPAs etc.)

20

4. Changes in equity and non-discrimination

Access by marginalised groups to services

Greater involvement by all groups in meetings, actions etc.

Leadership profile

21

5. Changes in civil society and communities capacity

Setting up of committees, frequency of meetings

Engagement in policy and programming Influencing work Growth in local NGOs/CBOs/civil

society action

22

Example 2:

Centre for Communication Programs, Johns Hopkins Bloomberg School of Public Health

Steps to Health Behaviour Change Indicators

23

Knowledge

– Recalls specific messages – Understands what messages means – Can name products, methods, or other practices and/or sources of services/supplies

Approval– Responds favorably to messages – Discusses messages or issues with members of personal networks (family, friends). – Thinks family, friends, and community approve of practice – Approves practice

Intention– Recognizes that specified health practices can meet a personal need – Intends to consult a provider – Intends to practice at some time

Practice– Goes to a provider of information/supplies/services – Chooses a method or practice and begins use – Continues use

Advocacy– Experiences and acknowledges the benefits of practice – Advocates the practice to others – Supports programs in the community

24

3. Collation and dissemination

25

Collecting the right data to measure your right indicators

Skills in participatory research and data collection

Focus Group Discussions Observation PRA Baseline survey

Other? 

26

Conducting the right dissemination of the right analysis of the right data on your right indicators

Using qualitative methods brings added responsibilities in terms of ensuring that collection is not merely extractive. How are you going to feedback to people who have put substantial time into the process?

27

Exercise: Focus Group Discussions The process for collating data and measuring change

is as important as developing the ‘right’ indicators. Exercise….

– Researchers: Lead the focus group discussion, attempting to ensure all participants are able to participate if appropriate. You are trying to find out about people’s attitudes and behaviours. You may choose to focus on children’s participation or on HIV/AIDS. This type of research would form a baseline for you to develop indicators and objectives to measure changes to attitudes, awareness, behaviours, practices, norms and values.

– Witnesses: To witness the session, and to be prepared to comment on the observed interaction

– Focus Group participants: Each participant will be assigned a role, as defined below.

28

4. Theory into Practice:Thematic Case Study work

29

Identifying, collecting, analysing and disseminating the right

indicators Which indicators do you want to collect?

How will you collect them?

How will you analyse them?

How will you disseminate them?

30

5. Theory into Practice:Peer Panel Review

31

1 or 2 people from each group stay at the stall to «sell their product» and to receive feedback. Rest rotate through other stalls every 10-15 minutes.

Members from the visiting groups examine the product posted at that stall and make comments on cards:- Suggestions for addition and/or reformulation

When the rotations to each stall are completed, the original small groups resume at their stall and discuss the received comments and incorporate suggestions into their work

Each group has 2 minutes to present back their changes in plenary.

32

6. Commitments and Close