measuring up: hiv-related advocacy evaluation training for civil society organisations...

Upload: kataisee-richardson

Post on 06-Apr-2018

216 views

Category:

Documents


0 download

TRANSCRIPT

  • 8/2/2019 Measuring up: HIV-related Advocacy Evaluation Training for Civil Society Organisations (facilitator's guide)

    1/38

    International HIV/AIDS Alliance

    (International secretariat)

    Preece House

    91-101 Davigdor Road

    Hove, BN3 1RE

    UK

    Telephone: +44(0)1273 718900

    Fax: +44(0)1273 718901

    [email protected]

    www.aidsalliance.org

    International Council o AIDS

    Service Organizations (ICASO)

    65 Wellesley Street East

    Suite 403

    Toronto, Ontario

    M4Y 1G7 Canada

    Telephone: +1(416)921 0018

    Fax: +1(416)921 9979

    [email protected]

    www.icaso.org

    MEASURING UPA GUIDE FOR FACILITATORS

    HIV-related advocacy evaluation training

    or civil society organisations

    AVFG 06/10

    acy facilitator cover.indd 1-2 22/10/2010 10:14

  • 8/2/2019 Measuring up: HIV-related Advocacy Evaluation Training for Civil Society Organisations (facilitator's guide)

    2/38

    ACKNOWLEDGEMENTS

    This guide or civil society organisations evaluating

    HIV-related advocacy was made possible through

    the contribution o many people who accepted being

    interviewed and who participated during the eld test o

    the guide:

    Agust Eendy Bin Abd Rachim Malaysian AIDS Council

    (MAC); Alan Msosa AIDS and Rights Alliance or

    Southern Arica (ARASA); D. Dhanikachalam Alliance

    India in Andhra Pradesh;. Dozie Ezechukwu Network

    o People Living with HIV/AIDS in Nigeria (NEPWHAN);

    Innocent Laison Arican Council o AIDS Service

    Organizations (AriCASO); Irene Naserian Kenya AIDS

    NGOs Consortium (KANCO); Jasmin Jalil Malaysian

    AIDS Council; Juan Jacobo Hernandez Colectivo Sol,

    Mexico; Ken Morrison Futures Group International;

    Meera Mishra (Consultant); Padma Buggineni Alliance

    India secretariat; Pavlo Skala Alliance Ukraine; P. S.

    Renuka Alliance India in Andhra Pradesh; RosemaryMburu Kenya AIDS NGOs Consortium (KANCO);

    Shaleen Rakesh Alliance India secretariat; Sunita

    Grote Alliance India secretariat; Valerie Pierre Pierre

    International Council o AIDS Service Organizations

    (ICASO).

    The International Council o AIDS Service Organizations

    (ICASO) and the International HIV/AIDS Alliance (Alliance)

    would like to thank the Joint United Nations Programme

    on HIV/AIDS (UNAIDS), the Canadian International

    Development Agency (CIDA) o the Government o

    Canada, Positive Action, the Ford Foundation and the Bill

    and Melinda Gates Foundation or their nancial support.

    The contents are the responsibility o ICASO and the

    Alliance and do not necessarily refect the views o any o

    the aorementioned institutions.

    Written by Nicky Davies (Davies & Lee: AIDS and

    Development Consulting, www.aidsdev.com) and

    Alan Brotherton (Alliance secretariat).

    Design by Jane Shepherd.

    Coordination by Kieran Daly and Mary Ann Torres (ICASO)

    and Barbara Pozzoni, Claude Cheta and Anton Kerr

    (Alliance).

    Copyright 2010 International HIV/AIDS Alliance and the

    International Council o AIDS Service Organizations.

    Inormation and illustrations contained in this publication

    may be reely reproduced, published or otherwise used

    or non-prot purposes without permission rom the

    Alliance or ICASO. However, the Alliance and ICASO

    request they be cited as the source o the inormation.

    ISBN: 1-905055-65-X

    Published: July 2010

    International HIV/AIDS Alliance

    Established in 1993, the International HIV/AIDS All iance (the

    Alliance) is a global alliance o nationally-based organisations

    working to support community action on AIDS in developing

    countries. To date we have provided support to organisations

    rom more than 40 developing countries or over 3,000

    projects, reaching some o the poorest and most vulnerable

    communities with HIV prevention, care and support, and

    improved access to HIV treatment.

    The Alliances national members help local community

    groups and other NGOs to take action on HIV, and are

    supported by technical expertise, policy work, knowledge

    sharing and undraising carried out across the All iance. In

    addition, the Alliance has extensive regional programmes,

    representative oces in the USA and Brussels, and works on

    a range o international activities such as support or South-

    South cooperation, operations research, training and good

    practice programme development, as well as policy analysis

    and advocacy.

    For more inormation about Alliance publications, please go

    to: www.aidsalliance.org/publications

    Registered charity number 1038860.

    The International Council of AIDS Service

    Organizations (ICASO)

    Founded in 1991, the International Council o AIDS Service

    Organizations (ICASO) mission is to mobilise and support

    diverse community organisations to build an eective global

    response to HIV and AIDS. This is done within a vision o

    a world where people living with and aected by HIV and

    AIDS can enjoy lie ree rom stigma, discrimination, and

    persecution, and have access to prevention, treatment

    and care.

    The ICASO network o networks operates globally, regionally

    and locally, and reaches over 100 countries around the world.

    The International Secretariat o ICASO is in Canada and its

    Regional Secretariats are based in Arica, Asia and the Pacic,

    Europe, Latin America and the Caribbean, and North America.

    For more inormation about ICASO and its Regional

    Secretariats, please go to: www.icaso.org

    acy facilitator cover.indd 3-4 22/10/2010 10:14

  • 8/2/2019 Measuring up: HIV-related Advocacy Evaluation Training for Civil Society Organisations (facilitator's guide)

    3/38

    1MEASURING UP \\A GUIDE FOR FACILITATORS

    What is the guide and how can it be used?

    The guide has two main components:

    1. a stand-alone learners guide or anyone

    interested in evaluating HIV-related advocacy

    2. a acilitators guide that describes how to runa two-day skills-building workshop to cover

    the content o the learners guide and apply

    the skills learned. Accompanying the learnersguide (the main handout and reerence

    material or sessions), there are ten PowerPoint

    presentations and a document or acilitators

    called Measuring up: workshop supplementary

    acilitation material.

    Who is the guide written or?

    The guide is a resource or leaders, advocacy and

    monitoring and evaluation sta o civil society

    organisations (including networks) who are

    involved in designing, implementing and assessing

    advocacy projects at dierent levels international,national and sub-national. Leaders o networks

    o key populations are likely to nd this guideparticularly relevant and helpul.

    What is the purpose o the guide?

    The overall purpose is to increase users capacity to

    evaluate the progress and results o their advocacy

    work.

    The guide aims to:

    1. help users to identiy and conront the

    challenges aced by community-basedorganisations evaluating HIV-related advocacy

    2. introduce new thinking or designing advocacy

    evaluations

    3. give users the opportunity to apply some

    aspects o the evaluation design process to their

    specic contexts

    4. make users aware that advocacy evaluation is

    a ast-growing and evolving eld, with a large

    number o publications on advocacy evaluation

    design, approaches and methods available via

    the Internet and summarised in the resources

    section o the learners guide.

    Why has the guide been developed?

    During the XVII International AIDS Conerence

    in Mexico in August 2008, the Alliance, ICASO

    and Constella Futures jointly organised a skills-

    building session on the challenges o monitoring

    and evaluating advocacy work. There was a high

    level o interest, and participants wanted to acquire

    the practical skills necessary to improve their

    monitoring and evaluation. So the acilitation teamagreed to explore dierent ways o responding to

    this demand and ollow up on lessons learned romMexico.

    Until recently, ew resources existed to guide

    evaluation in this area. However, in the last

    ew years advocacy evaluation has become a

    burgeoning eld, and several approaches and

    tools or evaluating advocacy and policy work

    have been developed. Thereore, the Alliance and

    ICASO decided to invest in producing a guide to

    evaluating advocacy work, drawing on the dierent

    approaches and tools now available.

    Who contributed to the content o the guide?

    The guide includes numerous extracts rom

    publications developed by leading organisations

    in the eld o advocacy evaluation, includingOrganizational Research Services, Harvard Family

    Research Project and Innovation Network. Any

    unintended misrepresentation o inormation rom

    these sources is the ault o the guide developers.

    Other key contributors to the development o

    the guide were ICASO and Alliance sta, ICASO

    and Alliance grantees and network memberorganisations, and consultant Nicky Davies

    (www.aidsdev.com).

    MEASURING UP HIV-related advocac evaluationtraining for civil societorganisations

    INTRODUCTION TO THE GUIDE

    A GUIDE FOR FACILITATORS

  • 8/2/2019 Measuring up: HIV-related Advocacy Evaluation Training for Civil Society Organisations (facilitator's guide)

    4/38

    2MEASURING UP \\A GUIDE FOR FACILITATORS

    How to use the acilitators guide

    The acilitators guide suggests activities or a three-

    day skills-building workshop and one-day learners

    workshop that aim to introduce the content and

    key messages o the learners guide. The workshop

    aims to build on the existing capacity o civil society

    organisation sta evaluating their advocacy work

    on HIV-related issues in resource-limited settings.We have also suggested an agenda or a one-day

    workshop that can be customised depending on the

    needs, interests and expertise o the group.

    Beore you acilitate the workshop, make sure thatyou are amiliar with the content o the learners

    guide and its key messages. Accompanying the

    acilitation guidance in this document are ten

    PowerPoint presentations or your use. The content

    o the PowerPoint presentations mirrors that o the

    learners guide and includes suggested presentation

    points within the session instructions in this guide.

    The PowerPoint presentations are deliberately

    dense with inormation. You should edit them to

    suit your presentation style and level o condence.

    The more text you edit, the more you will need to

    include this inormation condently in what you say

    to support the slide.

    This acilitators guide provides detailed guidance

    on how to acilitate a three-day or one-day

    workshop. The workshop can be adapted to

    be longer than three days by extending the

    time or key activities and discussion time ater

    presentations and group work. You could also

    provide extra time or participants to plan their

    own evaluations and make presentations to othersor eedback. Alternatively, you might want to

    allocate time or participants to share examples o

    previous evaluation work, discuss the strengths and

    limitations o the approaches they have taken, andlearn about the data collection methods and data

    tracking tools that dierent organisations use. A

    suggested three-day workshop timetable is on page

    4 and a one-day workshop timetable is on page 5.

    This guide has been written with 14 to 24

    participants in mind. However, i you have more or

    ewer participants than this, be sure to adapt the

    acilitation methods and instructions accordingly.We suggest that you invite two people rom each

    organisation to attend a three-day skills-building

    workshop using this guide. One person should

    have strong advocacy experience and one person

    should have strong monitoring and evaluationexperience. This will make sure that the group

    work sessions are well inormed. I two people

    rom each organisation are unable to attend the

    workshop then you will need to amend some o the

    participatory learning activities or the sessions.

    There are a number o publications available

    to help you to plan and acilitate a successul

    workshop. I you eel like a acilitation

    reresher you can review the ollowingpublications:

    A acilitators guide to participatory

    workshops with NGOs/CBOs responding toHIV/AIDS(International HIV/AIDS Alliance,

    2001)

    100 ways to energise groups: games

    to use in workshops, meetings and the

    community(International HIV/AIDS

    Alliance, 2002)

    Training and practice manuals 1: how to

    plan and run a participatory workshop

    (Southern Arican AIDS Trust, 2004)

    The Measuring uplearners guide shouldbe handed out to participants at the start o

    the workshop. This will provide important

    materials or your acilitation. Extra acilitation

    material is included in the Measuring up:

    workshop supplementary acilitation material

    document (in Microsot Word or easy

    amendment).

  • 8/2/2019 Measuring up: HIV-related Advocacy Evaluation Training for Civil Society Organisations (facilitator's guide)

    5/38

    3MEASURING UP \\A GUIDE FOR FACILITATORS

    You will need the ollowing resources or

    your workshop:

    l Measuring upacilitators guide one per

    acilitator and one spare

    l

    Measuring uplearners guide one perparticipant and acilitator, and a ew spare

    l ten Measuring upworkshop PowerPoint

    presentations

    l printed content o the Measuring up: workshop

    supplementary acilitation materialas described

    in the session instructions

    l Evaluation design ramework fip chart showing

    the diagram o the seven components o the

    advocacy evaluation design ramework, rompage 11 o the learners guide (see also the

    acilitators notes or the intoductory session to

    Module 3, page 18)l computer with Adobe Acrobat Reader and

    PowerPoint packages, projector (check it talks

    to your computer), projector screen, extension

    cables and plug adapters

    l access to a printer to print o inormation

    rom Measuring up: workshop supplementary

    acilitation material

    l notepads, pens and workshop timetables (one

    set or each participants chair at the start o

    the workshop) the workshop timetable can

    be adapted and amended using the template

    in Measuring up: workshop supplementaryacilitation material

    l sticky labels or name tags with participants ull

    name and organisation encourage participants

    to add or underline the name they would like to

    be used during the workshop

    l fip chart stands (preerably two), plenty o fipchart paper and 30 fip chart pens (mainly black

    and blue, plus our to ve red)

    l any props or preparation needed or energisers

    (red boxes with ideas or an energiser are

    included at relevant points in the acilitators

    guide)

    l make a car park fip chart or any questions

    asked by participants during the workshop that

    you intend to address later

    l rereshments (including healthy snacks and drink

    options to maintain energy levels and support

    those taking medication), water available at

    all times, meals, accommodation and daily

    allowances or participants and acilitators

    l semi-circle o chairs or plenary sessions, with

    tables and chairs around the edge o the room

    and breakaway spaces to accommodate all

    participants or group work.

    Important note

    I you are acilitating a one-day workshop

    using the instructions provided in this guide,

    you will need to add inormation to some

    o the existing PowerPoint presentations

    or create fip charts o inormation. All the

    inormation you will need can be drawn romthe learners guide. Details o when you will

    need to do this are included in the one-day

    workshop instructions box at the end o each

    session described in this guide.

  • 8/2/2019 Measuring up: HIV-related Advocacy Evaluation Training for Civil Society Organisations (facilitator's guide)

    6/38

    4MEASURING UP \\A GUIDE FOR FACILITATORS

    Suggested three-da orshop timetableMeasuring up: HIV-related advocac training for civil societ organisations

    DAYONE

    Timing Activity

    09:00 09:30 Ice-breaker and workshop expectations

    09:30 10:00 Workshop introduction, ground rules and pre-workshop questionnaire

    10:00 10:05 Session 1: Module 1 introduction

    10:05 10:30 Session 1.1: Dening advocacy

    10:30 11:00 Rereshments break

    11:00 12:00 Session 1.2: Dening key elements o monitoring and evaluation

    12:00 12:05 Session 2: Module 2 introduction

    12:05 12:40 Session 2.1: Understanding why it is important to evaluate advocacy

    12:40 13:00 Flexible session

    13:00 14:00 Lunch break

    14:00 15:05 Session 2.2: Identiying the challenges aced by civil society organisations

    evaluating HIV-related advocacy

    15:05 15:30 Session 2.3: Exploring how evaluating advocacy can be dierent romprogramme evaluations

    15:30 16:00 Rereshments break

    16:00 16:15 Session 3: Module 3 introduction

    16:15 17:00 Session 3.1: Determining evaluation users and uses

    DAY

    TWO

    09:00 09:30 Day one recap

    09:30 10:30 Session 3.2(a): Mapping your advocacy work

    10:30 11:00 Rereshments break

    11:00 12:00 Session 3.2(b): Mapping your advocacy work

    12:00 12:05 Energiser

    12:05 13.00 Session 3.3: Prioritising what to evaluate

    13:00 14:00 Lunch break

    14:00 15:05 Session 3.4: Developing your evaluation questions

    15:05 15:30 Session 3.5: Deciding on an approach to measurement

    15:30 16:00 Rereshments break

    16:00 17:00 Session 3.6: Selecting indicators

    DAYTHREE

    09:00 09:30 Day two recap

    09:30 10:30 Session 3.7: Identiying and choosing data collection methods relevant to

    advocacy evaluation

    10:30 11:00 Rereshments break

    11:00 12:00 Session 3.7: continued

    12:00 12:05 Session 4: Module 4 introduction

    12:05 13:00 Session 4.1: Consolidating learning on how to evaluate HIV-related advocacy

    13:00 14:00 Lunch break

    14:00 16:00 Session 4.1: continued

    16:00 16:15 Session 4.2: Refecting on the key messages and learning points rom the

    workshop to share with our colleagues

    16:15 17:00 Workshop summary, post-workshop questionnaire and workshop close

  • 8/2/2019 Measuring up: HIV-related Advocacy Evaluation Training for Civil Society Organisations (facilitator's guide)

    7/38

    5MEASURING UP \\A GUIDE FOR FACILITATORS

    Suggested one-da orshop timetableMeasuring up: HIV-related advocac training for civil societ organisations

    Timing Activity

    09:00 09:20 Workshop introduction

    09:20 09:30 Session 1: Module 1 introduction and Session 1.1: Dening advocacy

    09:30 09:50 Session 1.2: Dening key elements o monitoring and evaluation

    09:50 10:00 Session 2: Module 2 introduction and Session 2.1: Understanding why it is

    important to evaluate advocacy

    10:00 10:15 Session 2.2: Identiying the challenges aced by civil society organisations

    evaluating HIV-related advocacy

    10:15 10:30 Session 2.3: Exploring how evaluating advocacy can be dierent romprogramme evaluations

    10:30 10:50 Rereshments break

    10:50 11:00 Session 3: Module 3 introduction

    11:00 11:05 Session 3.1: Determining evaluation users and uses

    11:05 11:35 Session 3.2: Mapping your advocacy work

    11:35 11:45 Session 3.3: Prioritising what to evaluate

    11:45 11:55 Quick energiser

    11:55 12:05 Session 3.4: Developing your evaluation questions

    12:05 12:15 Session 3.5: Deciding on an approach to measurement

    12:15 12:35 Session 3.6: Selecting indicators

    12:35 13:15 Session 3.7: Identiying and choosing data collection methods relevant to

    advocacy evaluation

    13:15 14:00 Lunch break

    14:00 16:30 Session 4: Module 4 introduction and 4.1: Consolidating learning on how to

    evaluate HIV-related advocacy

    16:30 16:50 Session 4.2: Refecting on the key messages and learning points rom the

    workshop to share with our colleagues

    16:50 17:00 Workshop summary, post-workshop questionnaire and workshop close

  • 8/2/2019 Measuring up: HIV-related Advocacy Evaluation Training for Civil Society Organisations (facilitator's guide)

    8/38

    6MEASURING UP \\A GUIDE FOR FACILITATORS

    Ice-breaker and workshop expectations 7

    Workshop introduction, ground rules and pre-workshop questionnaire 7

    Module 1 Defning advocacy and key elements o monitoring and evaluation 9

    Session 1.1 Dening advocacy 10

    Session 1.2 Dening key elements o monitoring and evaluation 11

    Module 2 Understanding why evaluating HIV-related advocacy is important, 12

    its challenges and dierences

    Session 2.1 Understanding why it is important to evaluate advocacy 13

    Flexible session 14

    Session 2.2 Identiying the challenges aced by civil society organisations 15

    evaluating HIV-related advocacy

    Session 2.3 Exploring how evaluating advocacy is dierent rom

    programme evaluations 17

    Module 3 Introducing a ramework or designing evaluations o 18

    HIV-related advocacy

    Session 3.1 Determining evaluation users and uses 19

    Session 3.2 Mapping your advocacy work 20

    Session 3.3 Prioritising what to evaluate 23

    Session 3.4 Developing your evaluation questions 24

    Session 3.5 Deciding on an approach to measurement 25

    Session 3.6 Selecting indicators 26

    Session 3.7 Identiying and choosing data collection methods relevant to 27

    advocacy evaluation

    Module 4 Putting it all into action 29

    Session 4.1 Consolidating learning on how to evaluate HIV-related advocacy 29

    Session 4.2 Refecting on the key messages and learning points rom the 34

    workshop to share with our colleagues

    Closing session Workshop summary, post-workshop questionnaire and 36

    workshop close

    CONTENTS

  • 8/2/2019 Measuring up: HIV-related Advocacy Evaluation Training for Civil Society Organisations (facilitator's guide)

    9/38

    7MEASURING UP \\A GUIDE FOR FACILITATORS

    DAY ONE

    Facilitators notes

    A simple and quick ice-breakeris to ask participants to move

    around the room and greet

    each other, sharing their names,

    organisation and something

    dierent, interesting or sillyabout themselves (such as a

    way to remember their name).

    You can also do this sitting in the

    plenary chairs, which is quicker

    but more serious.

    Materials

    A small piece o paper and apen, one or each participant

    Time

    30 minutes

    Instructions

    1. Welcome everyone to the workshop.2. Facilitate a 15-minute Ice-breaker to help everyone to get to

    know each other.

    3. Facilitate a 15-minute activity to understand participants

    expectations o the workshop. Hand out small pieces o paper

    and marker pens to each participant. Ask participants to spend

    ve minutes brainstorming their expectations o the workshopand write each one on a separate small piece o paper. Ater

    ve minutes ask participants to volunteer to read out their

    expectations. Each participant can oer one expectation at a

    time until they have all been read out. As each expectation is

    read aloud, collect the piece o paper and stick it up or all to

    see. I others have similar expectations, ask i you can collect

    them and stick them in a group. Once all the expectations have

    been collected, review them and explain which ones will or will

    not be addressed during the workshop and why. I realistic and

    i you eel condent to do so, explain which expectations you

    will try to add to the workshop sessions or timetable.

    Facilitators notes

    l You can use Measuring up workshop PowerPoint 1

    Measuring up workshop introduction or this workshop

    introduction. It is also useul to have fip charts with the

    workshops purpose, aims and ground rules up on the

    workshop room walls. Make sure that participants havethe workshop timetable on their chairs at the start. You can

    acilitate developing the ground rules i you think this is

    necessary and you can accommodate the time needed to

    do so.

    l I you eel that you have time, you could acilitate a brie

    exercise (510 minutes) to gather the expectations o

    participants. Remember to give participants eedback on which

    expectations you aim to meet during the workshop and why.

    OPENING SESSION Ice-breaker and workshop expectations

    Workshop introduction, ground rules and pre-workshop

    questionnaire

    Session inormation

    The purpose o the guide,and thereore this workshop,

    can be ound on page 3 o the

    introduction to this acilitators

    guide, and the suggested

    workshop timetables or three-

    or one-day workshops can be

    ound on pages 4 and 5.

    Materials

    l PowerPoint 1

    l Workshop timetable, one or

    each participant

    l Copies o the pre- and post-

    workshop questionnaire rom

    Measuring up: workshop

    supplementary acilitation

    material

    l Flip charts with workshop

    aims and ground rules

    Time

    30 minutes

  • 8/2/2019 Measuring up: HIV-related Advocacy Evaluation Training for Civil Society Organisations (facilitator's guide)

    10/38

    8MEASURING UP \\A GUIDE FOR FACILITATORS

    DAY ONE

    Suggested ground rules or participants and the acilitator during the workshop include:

    l Help each other to start and nish sessions and breaks on time.

    l Switch o mobile phones or leave them on vibrate (take phone calls during breaks where

    possible).

    l Respect each other by listening actively, providing constructive criticism and participating

    ully to support collective learning.

    l Recognise that there is a lot to learn and share; stay ocused on the topic and objectives o

    the sessions rather than be drawn into the details o broader issues.

    l Avoid using too much evaluation jargon there are diering levels o knowledge andexperience o evaluation work among participants and jargon may not be amiliar to

    everyone.

    Time

    20 minutes

    Instructions1. Welcome everyone to the workshop.

    2. Ask the participants and acilitator to say their name and the organisation they are rom.

    3. Give a 10-minute presentation introducing the guide and its two components; the workshoppurpose and aims; the workshop timetable; and basic ground rules or participants and acilitator.

    (Note you will need to add basic ground rules to Measuring up workshop PowerPoint 1

    Measuring up: workshop introduction or make a fip chart.)

    4. Hand out the learners guide to each participant and the pre- and post-workshop questionnaire.

    5. Ask participants to spend a ew minutes completing the rst questions on the pre-and post-

    workshop questionnaire. Since this will be an anonymous process, ask them to old the

    questionnaire in hal and draw a symbol or little picture on the back (not their name) so that they

    can identiy their own questionnaire at the end o the workshop.

    ONE-DAy wORkSHOP

    Instructions

    1. Give a 10-minute introduction to the training pack and its two

    components; the workshop purpose and aims; and provide a

    summary review o the workshop timetable.

    2. Spend ten minutes brainstorming and agreeing some basic

    ground rules or participants and acilitator on fip chart paper

    and stick them to the workshop wall.

    3. Ask participants to spend ve minutes completing the rst

    questions on the pre- and post-workshop questionnaire.

    Explain that the questionnaire will be repeated at the end

    o the workshop. The data will be anonymous and the

    questionnaire will be kept sae and not read in the meantime.

    Then ask them to old the questionnaire in hal and draw a

    symbol or little picture on the back (not their name) so that

    they can identiy their own questionnaire at the end o the

    workshop. Collect the questionnaires.

    4. Hand out the learners guide to each participant and ask them

    to write either their own name or their organisations name onthe guide. Tell them to have this guide always available during

    session time.

  • 8/2/2019 Measuring up: HIV-related Advocacy Evaluation Training for Civil Society Organisations (facilitator's guide)

    11/38

    9MEASURING UP \\A GUIDE FOR FACILITATORS

    DAY ONE

    MODULE 1Dening advocacy and key elements o monitoring

    and evaluation introduction to Module 1

    Session inormation

    There is rarely one correct orbest denition or dierent

    aspects o our work, and

    there are many denitions o

    advocacy and monitoring and

    evaluation. Here we introduce

    one advocacy denition

    and important elements o

    monitoring and evaluation,

    along with some examples to

    help clariy key terms.

    Facilitators notes

    You can use Measuring upworkshop PowerPoint 2

    Module 1: Defning advocacy

    and key elements o monitoring

    and evaluation or this

    introduction, although it is also

    useul to have a fip chart o the

    module objectives taped to the

    wall during sessions.

    Materials

    l PowerPoint 2

    l

    Flip chart

    Time

    5 minutes

    Instructions

    Give a brie introduction to this module, explaining the objectivesor the session.

    Objectives or Module 1

    l To provide a useul denition o advocacy.

    l To provide reresher inormation on thedenition o monitoring and evaluation, the

    distinction between outputs and outcomes,

    and the principles underlying how they can be

    measured.

  • 8/2/2019 Measuring up: HIV-related Advocacy Evaluation Training for Civil Society Organisations (facilitator's guide)

    12/38

    10MEASURING UP \\A GUIDE FOR FACILITATORS

    DAY ONE

    SESSION 1.1 Dening advocacy

    Session inormation

    A denition o advocacy and anexplanation o its key elementsor characteristics are included

    on page 5 o the learners guide.

    Facilitators notes

    You can use Measuring up

    workshop PowerPoint 2

    Module 1: Defning advocacy

    and key elements o monitoring

    and evaluation to present the

    advocacy denition and key

    elements.

    Materials

    PowerPoint 2

    Time

    25 minutes

    Instructions

    1. Introduce the name o the session and its place in theworkshop timetable.

    2. Ask participants to brainstorm the key points or dening

    advocacy.

    3. Present the advocacy denition and its key elements or

    characteristics.

    4. Ask participants i they know o other good advocacy

    denitions.

    5. Conclude the session by repeating that there is not one correct

    denition and that there is urther guidance available in the

    APCASO toolkit.

    6. Tell participants that there is now a 30-minute break orrereshments and what time they should be back in their

    chairs.

    Time

    10 minutes

    Instructions

    1. Present the advocacy denition and its key elements or characteristics.

    2. Ask participants i they know o other good advocacy denitions.

    3. Conclude the session by repeating that there is not one correct denition and that there is urther

    guidance available in the APCASO toolkit.

    ONE-DAy wORkSHOP

  • 8/2/2019 Measuring up: HIV-related Advocacy Evaluation Training for Civil Society Organisations (facilitator's guide)

    13/38

    11MEASURING UP \\A GUIDE FOR FACILITATORS

    DAY ONE

    Evaluation

    l Comparison o end to beginning

    l Results (outcomes and impact) ocus

    l Post-mortem

    l Assessing what has been achieved

    l Finding out what worked

    l Learning how to improve

    l Best done by an independent assessor

    SESSION 1.2 Dening key elements o monitoring and evaluation

    Session inormation

    Section 1.2 o the learners

    guide (pages 5 and 6) provides

    a number o denitions omonitoring and evaluation, the

    distinction between outputs and

    outcomes, and the principles

    underlying how they can be

    measured.

    Materials

    l Two fip chart papers with

    Monitoring written on the

    top o one and Evaluation

    on the otherl Tape

    l PowerPoint 2

    Time

    60 minutes

    Facilitators notes

    l Write monitoring at the top o a piece o fip chart paper andevaluation at the top o another in advance o the session.

    Make sure you have tape ready.

    l You may want to use or adapt Measuring up workshopPowerPoint 2 Module 1: Defning advocacy and key elements

    o monitoring and evaluation to present the key denitions and

    examples rom the learners guide.

    l Try not to get drawn into a broader discussion about

    monitoring and evaluation. I you park ideas, you should say

    when and how you will address them at a later stage in the

    workshop (or example, the day two fexible session). I they

    are beyond the scope o the workshop you will need to explain

    this, but be as helpul as possible by reerring participants to asource o inormation or by talking over issues at lunchtime or

    during breaks.

    Instructions

    1. Introduce the name o the session and its place in the

    workshop timetable.

    2. Tape fip charts with monitoring and evaluation written at

    the top to a wall or fip chart stand. Ask participants: Can

    you brainstorm the key points or defning monitoring and

    evaluation?

    3. Explain that you are going to present some denitions o

    monitoring and evaluation, the distinction between outputsand outcomes, and the principles underlying how they can

    be measured. Give the presentation. Read the denitions o

    monitoring and evaluation slowly, and repeat i necessary.

    4. Ask participants i they have any questions about the

    presentation. Address the questions and add any that you

    cant answer or that would be better addressed later in the

    workshop on the car park fip chart.

    Brainstorm example: Can you brainstorm the key points or dening monitoring and evaluation?

    Monitoring

    l Tracking activities

    l Process ocus

    l Documenting what has happened

    l Recording activity outputs

    l Checking were on track

    l Observing implementation

    l Done in house

  • 8/2/2019 Measuring up: HIV-related Advocacy Evaluation Training for Civil Society Organisations (facilitator's guide)

    14/38

    12MEASURING UP \\A GUIDE FOR FACILITATORS

    DAY ONE

    MODULE 2Understanding why evaluating HIV-related advocacy is

    important, and conronting the challenges introduction

    to Module 2

    Facilitators notes

    Prepare a fip chart with the

    objectives or Module 2 and tape

    it to the wall as the visual aid or

    your introduction.

    Materials

    Flip chart

    Time

    5 minutes

    Instructions

    Give a brie introduction to this module, explaining the objectives

    or the session.

    Time

    20 minutes

    Instructions

    1. Explain that you are going to present some denitions o monitoring and evaluation, the

    distinction between outputs and outcomes, and the principles underlying how they can bemeasured. Give the presentation.

    2. Ask participants i they have any questions about the presentation. Address these quickly or addthem to the car park fip chart to address in later sessions, during breaks with individuals or at the

    end o the day.

    ONE-DAy wORkSHOP

    Objectives or Module 2

    l To share why it is important to evaluateadvocacy.

    l To identiy the challenges o evaluating our

    HIV-related advocacy.

    l To understand how evaluating advocacy is

    dierent rom programme evaluation.

  • 8/2/2019 Measuring up: HIV-related Advocacy Evaluation Training for Civil Society Organisations (facilitator's guide)

    15/38

    13MEASURING UP \\A GUIDE FOR FACILITATORS

    DAY ONE

    There is growing interest in advocacy evaluation, both rom advocates and unders. The reasons or

    this interest may dier between the two groups, but in some cases there may be overlap. The main

    reasons or evaluating advocacy include a desire to:

    l learn how to improve the capacity o advocates

    l learn how to adjust advocacy strategies as they are being implemented

    l inorm the planning o uture advocacy work, including unding cycles/proposals

    l demonstrate relative worth and specic areas o added value (or example, or networks o key

    populations 1 that are competing or unding with development non-governmental organisationsand academic institutions)

    l demonstrate evidence-based approaches to advocacy work (increasingly demanded by unders,

    infuential policymakers and communities to encourage engagement)

    l account or unding and demonstrate results

    l demonstrate results to mobilise more resources or uture advocacy work, including monitoring

    and evaluation o advocacy.

    SESSION 2.1 Understanding why it is important to evaluate advocacy

    Session inormation

    The introductory text below hasbeen extracted rom page 7 o

    the learners guide.

    Facilitators notes

    Circulate among the groups to

    check they are not stuck on the

    task.

    Materials

    Flip chart

    Time

    35 minutes

    Instructions

    1. Introduce the name o the session and its place in theworkshop timetable.

    2. Ask participants to orm mini groups o three people with their

    neighbours by adjusting their chairs slightly. Ask groups: Can

    you identiy the main reasons or evaluating advocacy work?

    Tell them they have ve minutes to consider the question in

    their groups.

    3. Ater ve minutes ask participants to move their chairs

    back. Tell them you would like to collect their ideas ask or

    dierent contributions rom participants. Write the ideas on a

    fip chart.

    4. Check that the main reasons identied below (see page 7 o

    the learners guide)have been considered.

    1. The term key populations reers to groups o individuals particularly aected by HIV, including people living with HIV, sex workers, men who havesex with men, people with disabilities, people who use drugs, prisoners, etc.

    Time

    10 minutes

    Instructions

    1. Present the main reasons why it is important to conduct advocacy evaluation, provided in the

    session inormation above.

    2. Ask i the participants have anything to add.

    ONE-DAy wORkSHOP

  • 8/2/2019 Measuring up: HIV-related Advocacy Evaluation Training for Civil Society Organisations (facilitator's guide)

    16/38

    14MEASURING UP \\A GUIDE FOR FACILITATORS

    DAY ONE

    FLEXIBLE SESSION

    Session inormation

    This session can be titledand used as necessary by the

    acilitator. For example, this

    could be let blank to allow or

    session overrun, be titled car

    park to address any questions

    that have not be addressed, or

    be titled administration session

    and used to hand out per diems,

    address logistics and anyissues concerning participants.

    Alternatively you could conduct

    a un energiser i appropriate.

    Time

    20 minutes

    Instructions

    1. To be determined by the acilitator.2. Tell participants that there is now a one-hour break or lunch

    and what time they should be back in their chairs.

    Idea or an energiser

    Simon says

    The acilitator tells the group that they should ollow any

    instructions that begin Simon says. I the acilitator does

    not begin an instruction with the words Simon says then

    the group should not ollow the instruction. The acilitatorbegins by saying something like, Simon says clap your

    hands, while clapping her hands. The participants ollow.

    The acilitator speeds up the actions, always beginning

    with the words Simon says. Ater a short while the words

    Simon says are omitted. Those participants who continue

    to ollow the instructions are out o the game. The game can

    carry on or as long as it remains un.

    Source: 100 ways to energise groups: games to use in workshops, meetings and thecommunity (International HIV/AIDS Alliance, 2003)

    This session is not relevant or a one-day workshop.

  • 8/2/2019 Measuring up: HIV-related Advocacy Evaluation Training for Civil Society Organisations (facilitator's guide)

    17/38

    15MEASURING UP \\A GUIDE FOR FACILITATORS

    DAY ONE

    SESSION 2.2Identiying the challenges aced by civil society

    organisations evaluating HIV-related advocacy

    Session inormation

    Key challenges identied byguide contributors can be ound

    on pages 7 and 8 o the learners

    guide.

    Materials

    One set o the below or each

    group:

    l Challenges rom the learners

    guide, printed o rom the

    Measuring up: workshop

    supplementary acilitationmaterialand cut into strips

    (make sure you cut the blankends o the strips so that

    they can t side by side on

    the scale)

    l Tape (Stickie Stu or Blu-Tak

    etc.)

    l A ew ordinary (not fip chart)

    pens

    l 20 spare strips o paper the

    same size as the challenges

    (enough or additional

    challenges and ways to avoid

    or overcome challenges)

    l Instructions or the activity

    printed o rom the workshop

    supplementary acilitation

    materials

    l Flip chart paper with the

    sliding-scale presented

    horizontally at the top and

    across the ull width o the

    paper so that the challenges

    can be placed to the right

    Time

    65 minutes

    Facilitators notes

    l Prepare resources or group work in advance dont leave this

    to break-time.

    l Prepare group work spaces in advance, including tables and

    correct number o chairs, to avoid wasting time during the

    session.

    l Circulate among the groups to help them be productive,ocused and not get stuck.

    l Keep to time, even i participants havent completed all the

    activity instructions.

    l The learners guide notes that challenges that are more within

    our control are easier to overcome than those more out o ourcontrol. This could be a useul point o discussion ater the

    activity.

    Instructions

    1. Introduce the name o the session and its place in the

    workshop timetable.

    2. Explain to participants that the learners guide lists challenges

    identied by training pack contributors to evaluating HIV-

    related advocacy work in resource-limited settings. Some o

    these challenges are specic to HIV; others to the capacity o

    civil society organisations and the context o resource-limited

    settings. The list is not exhaustive or in any particular order.

    3. Explain that you would like participants to work in groups o

    approximately our people rom any organisation. Read out

    the instructions on the next page and ask each group to collect

    rom you a fip chart, pens, a set o strips o challenges, spare

    strips, tape and activity instructions printed o or each group.Indicate where the group work spaces are.

    Participants conducting this activity at the Measuring up eld-test workshop, India 2010

  • 8/2/2019 Measuring up: HIV-related Advocacy Evaluation Training for Civil Society Organisations (facilitator's guide)

    18/38

    16MEASURING UP \\A GUIDE FOR FACILITATORS

    DAY ONE

    Group work instructions

    l You have 35 minutes to complete this activity.

    l Read through as a group the challenges identied by the

    guide contributors.

    l Write any other key challenges on additional strips o paper

    (dont spend too long on this or go into detail; you can alwaysadd challenges later).

    l Organise the challenges on the fip chart using the sliding-

    scale at the top o the page, sticking the challenges along the

    sliding-scale. Where you disagree among yourselves, you will

    need to average the score when placing the challenge, or use

    your own rule such as how most civil society organisations

    working on HIV would eel. Remember, this is a tool to

    acilitate discussion, so complete the activity rather than getstuck. Note: please write the names o your group members

    on your fip chart.

    l I you have time, write down on spare strips o paper ways to

    avoid or overcome the challenges that you have put in or nearthis end o the sliding scale.

    4. Ater 35 minutes ask the groups to place their group work fip

    charts together on the central area o the foor.

    5. Facilitate a discussion or the remainder o the session

    time around the outputs rom the groups, noting additional

    challenges, placement o challenges and ways to avoid or

    overcome challenges i there are any. Draw some conclusions

    about which challenges are easier to overcome and why, and

    which are more dicult to overcome and why.

    1 2 3 4

    Although perceived

    as a challenge it canbe easily avoided or

    overcome

    A challenge so

    great it means thatevaluating advocacy

    is almost impossible,

    meaningless or

    damaging

    Sliding scale

    Time

    15 minutes

    Instructions

    1. Ask the participants to orm buzz pairs with the person sitting next to them. Ask them to spend a

    couple o minutes sharing a ew challenges they ace in evaluating their HIV-related advocacy.

    2. Present a fip chart or PowerPoint presentation with the challenges aced by civil society

    organisations evaluating HIV-related advocacy rom pages 7 and 8 o the learners guide (you will

    need to prepare this fip chart or PowerPoint presentation).

    3. Ask the participants to share any additional challenges they identied in their buzz pairs.

    ONE-DAy wORkSHOP

  • 8/2/2019 Measuring up: HIV-related Advocacy Evaluation Training for Civil Society Organisations (facilitator's guide)

    19/38

    17MEASURING UP \\A GUIDE FOR FACILITATORS

    DAY ONE

    17MEASURING UP \\A GUIDE FOR FACILITATORS

    SESSION 2.3Exploring how evaluating advocacy can be dierent rom

    programme evaluations

    Session inormation

    Inormation or this session canbe ound on pages 8 to 10 o the

    learners guide.

    Materials

    PowerPoint 3

    Time

    25 minutes

    Facilitators notes

    l You may want to use or adapt Measuring up workshop

    PowerPoint 3 Module 2: Exploring how evaluating advocacy

    work is dierent rom programme evaluationsto give this

    presentation. You will need to know the page number o box 3

    in the learners guide.

    l I the discussion is productive or this session, you could

    break or rereshments late or continue the discussion ater

    rereshments, as session 3.1 can be shortened (see the

    acilitators notes or session 3.1).

    Instructions

    1. Introduce the name o the session and its place in theworkshop timetable.

    2. Explain to participants that you will be giving a presentation

    exploring how evaluating advocacy work is dierent rom

    evaluating programmes. Give the presentation.

    3. Ask i participants have any questions about the presentation

    and acilitate a discussion.

    4. Tell participants that there is now a 30-minute break or

    rereshments and what time they should be back in their

    chairs.

    Time

    15 minutes

    Instructions

    1. Present how evaluating advocacy can be dierent rom evaluating programmes.

    2. Ask i the participants have anyg questions or comments about the presentation. Discuss these.

    ONE-DAy wORkSHOP

  • 8/2/2019 Measuring up: HIV-related Advocacy Evaluation Training for Civil Society Organisations (facilitator's guide)

    20/38

    18MEASURING UP \\A GUIDE FOR FACILITATORS

    MODULE 3Introducing a ramework or designing evaluations o

    HIV-related advocacy introduction to Module 3

    Session inormation

    Inormation or this sessioncan be ound on page 11 o the

    learners guide.

    Facilitators notes

    l You can use Measuring

    up workshop PowerPoint

    4 Module 3: Module 3Introduction to give this

    presentation.

    l Prepare an Evaluation design

    ramework fip chart showing

    the diagram o the seven

    components o the advocacyevaluation design ramework

    rom page 11 o the learners

    guide.

    Time15 minutes

    Instructions

    1. Introduce the name o the session and its place in theworkshop timetable.

    2. Give a brie introduction to this module, explaining theobjectives, the seven components o the ramework

    represented by the topic headings, and why it is important

    not to consider these components as steps. Highlight the

    inormation used to develop this section o the learners guide

    and suggest that they might nd it helpul to reer to these

    sources in the uture.

    Objectives or module 3

    l To provide inormation and tools or the seven

    key components o a ramework or decision-

    making on how to evaluate HIV-related

    advocacy.

    l To practise developing a theory o change,

    developing evaluation questions and

    identiying indicators.

    Time

    10 minutes

    Instructions

    1. Present Measuring up workshop PowerPoint 4 Module 3: Module 3 Introduction.

    2. Ask i the participants have any questions or comments about the presentation. Discuss.

    ONE-DAy wORkSHOP

    DAY ONE

  • 8/2/2019 Measuring up: HIV-related Advocacy Evaluation Training for Civil Society Organisations (facilitator's guide)

    21/38

    19MEASURING UP \\A GUIDE FOR FACILITATORS

    DAY ONE

    SESSION 3.1 Determining evaluation users and uses

    Session inormation

    Inormation or this sessioncan be ound on page 12 o the

    learners guide.

    Facilitators notes

    I more discussion time was

    needed or session 2.3 you can

    miss out instruction 3.

    Materials

    Flip chart

    Time40 minutes

    Instructions

    1. Introduce the name o the session and its place in theevaluation design ramework fip chart.

    2. Use the fip charts developed during session 2.1 (and/or page

    7 o the learners guide) to review the main reasons or, or

    uses o evaluations.

    3. Ask participants to identiy the main audiences or evaluations.

    Write them on a fip chart. Make sure that they include the

    main audiences identied in the learners guide.

    4. Place this fip chart next to the fip chart developed during

    session 2.1 and briefy acilitate participants to match the

    main reasons or evaluations with these dierent audiences.

    Comment on the extent o the overlap.

    5. Explain the importance o considering who the primaryaudience or the evaluation will be, and the value o working

    with other audiences (particularly unders) to develop design

    evaluations.

    6. Close day one o the workshop by reminding participants what

    time they should be back in the workshop room tomorrowmorning. Tell them about evening logistics, and that they

    should remember to bring their learners guides with them in

    the morning.

    Time

    15 minutes

    Instructions

    1. Use the fip charts developed during session 2.1 (and/or page 7 o the learners guide) to review

    the main reasons or, or uses o evaluations.

    2. Ask participants to identiy the main audiences or evaluations. Write them on a fip chart. Make

    sure that they include the main audiences identied in the learners guide.

    ONE-DAy wORkSHOP

  • 8/2/2019 Measuring up: HIV-related Advocacy Evaluation Training for Civil Society Organisations (facilitator's guide)

    22/38

    20MEASURING UP \\A GUIDE FOR FACILITATORS

    DAY TWO

    DAy TwO

    SESSION 3.2

    Recap o day one

    Mapping your advocacy work

    Session inormation

    Inormation or this session can

    be ound on pages 12to 17 o

    the learners guide.

    Materials

    One set o the ollowing or each

    group:

    l Approximately 30 rectangles

    o blank paper (about aneighth o the size o a piece o

    A4; i.e. old completely in hal

    three times, unold and cut)

    l A ew ordinary (not fip chart)

    pens

    l A piece o fip chart paper

    l Tape (Stickie Stu, Blu-Tak,

    etc.)

    l Instructions or the activity

    printed o rom the

    workshop supplementary

    acilitation material

    Time

    2 hours

    Facilitators notes session 3.2(a)

    l You may want to use or adapt Measuring up workshop

    PowerPoint 5 Module 3: Mapping your advocacy workto

    give this presentation. You have a maximum o 30 minutes.

    l Leave plenty o time or the group work activity.

    l Prepare the group work resources in advance and ensure

    there is space to work, with a table and correct number o

    chairs (or foor, i appropriate).

    l Make sure that all acilitators are constantly moving rom

    one group to another to provide support and answer

    questions. It is a good idea to allocate specic groups to

    individual acilitators so that only one acilitator works with

    each group. Share any problems you are having (as group

    support acilitators) with each other and support each other as

    necessary.

    Instructions session 3.2(a)

    1. Introduce the name o the session and its place in the

    evaluation design ramework fip chart.

    2. Explain to participants that the aim o this activity is to

    amiliarise them with the content o the learners guide,which tells them how to map their work and provides three

    rameworks to help them do so. The aim is also to allow them

    a chance to internalise the inormation and practise using

    it. Explain that a strategic view o our advocacy work, which

    breaks the goals down into interim outcomes or incremental

    measures, helps to address the challenges and dierences

    concerning HIV-related advocacy evaluation made in Module 2.

    3. Ask participants: How many o you, or your civil society

    organisation colleagues, have developed a theory o change

    or your advocacy work?

    Instructions

    1. Facilitate an energiser.2. Provide a recap o the previous days sessions.

    Facilitators notes

    It is useul to use a combinationo the workshop timetable

    and fip charts rom output or

    plenary brainstorms to review

    key points rom the previous

    day.

    Time

    30 minutes

  • 8/2/2019 Measuring up: HIV-related Advocacy Evaluation Training for Civil Society Organisations (facilitator's guide)

    23/38

    21MEASURING UP \\A GUIDE FOR FACILITATORS

    DAY TWO

    4. Explain to participants that you will give a presentation on howto develop a theory o change, including a set o instructions,

    a completed example and list o example advocacy outcomes

    that they can use to help develop their own theory o change

    or their advocacy work. Explain that this is a airly long

    presentation that covers the inormation in section 3.2 (pages

    12 to 17) o the learners guide. Tell participants they will need

    to reer to their learners guide during the presentation. Give

    the presentation.

    5. Ater the presentation, consider how much time you have

    let or this session beore the rereshments break. Adjust the

    timings or the remainder o this session as appropriate.

    6. Explain that or the ollowing learning activity you will ask

    them to work with colleagues rom the same organisation.

    Explain that they should ollow the instructions that will be

    handed to them.

    Activity instructions:

    l You have 70 minutes to complete this activity.

    l Identiy one real advocacy goal or impact statement rom

    your advocacy work.

    l Using your learners guides, have the ollowing three

    rameworks or developing a theory o change open or

    reerence:

    l Six steps or developing a theory o change visual map

    (box 3, page 14)

    l an example theory o change (gure 3, page 15)

    l Categories and menu o outcomes or advocacy and

    policy work (table 2, pages 16 and 17).

    l Develop a theory o change outcome map similar to gure

    3 using the blank rectangles o paper and an ordinary (notfip chart) pen. You can place the rectangles on a piece o fipchart paper to create a transportable map. Dont be tempted

    to copy gure 3; instead use the tools to develop your own

    map. You do not necessarily need to nish mapping; the

    activity aims to amiliarise you with the content o the three

    rameworks and to practise using them. For example, you

    may only develop a so that chain or one strategy on your

    outcome map. Remember to ocus on outcomes in your so

    that chain dont end up listing activities or strategies.

    7. Ask participants to share what they are learning rom

    developing their theory o change.

    8. Break or 30 minutes or rereshments.

    Facilitators notes session 3.2(b)

    l Make sure that all acilitators are constantly moving rom one

    group to another to provide support and answer questions.

    l At the end o the session, or when convenient, collect together

    all the maps and store ready or session 4.1 on day three.

    Theory o change map

    Time

    90 minutes

  • 8/2/2019 Measuring up: HIV-related Advocacy Evaluation Training for Civil Society Organisations (facilitator's guide)

    24/38

    22MEASURING UP \\A GUIDE FOR FACILITATORS

    DAY TWO

    Instructions session 3.2(b)

    1. Explain to participants that because it is so important to have

    a good theory o change to be able to design an advocacy

    evaluation, we will continue to map out the theories o change

    and also add some assumptions.

    2. Introduce the activity instructions below.

    Activity instructions:

    l Continue to work on your theory o change or the next 80

    minutes.

    l Beore the end o the group work time, stand back rom your

    outcome map and identiy assumptions you have made about

    the context o the advocacy work. Some examples are given in

    Figure 3 on page 15 o the learners guide.

    l Save at least ten minutes to refect on the process o

    developing the theories o change. Use the opportunity

    to reinorce the key benets o mapping out a theory o

    change (both o the process itsel and o the resulting map),particularly i participants have ound the activity dicult. Asummary o the key benets o a theory o change includes:

    l important or agreeing high-level strategies and outcomes,and common agreement about impact or your evaluation

    work

    l an opportunity to learn what others are doing inside and

    outside o the organisation, rom the dierent perspectives

    o mapping participants

    l a useul decision-making tool or ongoing learning and

    improvement o strategies

    l a visual map to include in undraising proposals.

    Time

    30 minutes

    Instructions

    1. Explain to participants that you will give a presentation outlining some tips or developing

    evaluation questions and some examples.

    2. Ask participants i they have any questions. Discuss as necessary.

    ONE-DAy wORkSHOP

  • 8/2/2019 Measuring up: HIV-related Advocacy Evaluation Training for Civil Society Organisations (facilitator's guide)

    25/38

    23MEASURING UP \\A GUIDE FOR FACILITATORS

    DAY TWO

    SESSION 3.3 Prioritising what to evaluate

    Session inormation

    Inormation or this sessioncan be ound on page 18 o the

    learners guide.

    Facilitators notes

    You may want to use or adapt

    Measuring up workshop

    PowerPoint 6 Module 3:

    Prioritising what to evaluateto

    give this presentation.

    Time

    55 minutes

    Instructions

    1. Conduct an energiser or ve minutes.2. Introduce the name o the session and its place in the

    evaluation design ramework fip chart.

    2. Give a presentation covering why it is important to prioritise

    what to evaluate and provide some guidance in making this

    decision.

    3. Ask i there are any questions and discuss as necessary.

    4. Ask participants to go back into their organisation groups and

    decide who would be the end users or an evaluation o their

    advocacy work and what they would prioritise or evaluation.

    5. Ask participants i they ound this a straightorward exercise.

    Time

    10 minutes

    Instructions

    1. Explain to participants that you will give a presentation on how to prioritise what to evaluate.

    2. Ask participants i they have any questions. Discuss as necessary.

    ONE-DAy wORkSHOP

  • 8/2/2019 Measuring up: HIV-related Advocacy Evaluation Training for Civil Society Organisations (facilitator's guide)

    26/38

    24MEASURING UP \\A GUIDE FOR FACILITATORS

    DAY TWO

    SESSION 3.4 Developing your evaluation questions

    Session inormation

    Inormation or this sessioncan be ound on page 15 o the

    learners guide.

    Facilitators notes

    You may want to use or adapt

    Measuring up workshop

    PowerPoint 7 Module 3:

    developing your evaluation

    questionsor the presentations

    during this session.

    Time

    65 minutes

    Instructions

    1. Introduce the name o the session and its place in theevaluation design ramework fip chart.

    2. Ask participants to share their experience o setting evaluationquestions. Write on a fip chart any points that will be useul to

    reer back to later.

    3. Explain to participants that you will give a presentation

    outlining some tips or developing evaluation questions and

    some examples.

    4. Ask participants i they have any questions. Discuss as

    necessary.

    5. Ask participants to work in their organisation groups to

    brainstorm what their evaluation questions might be.

    6. Bring the groups back together to share what they have

    learned rom the activity.

    7. Facilitate an energiser or give the participants a short break.

    Idea or an energiser

    Fruit saladChoose ve ruits (or example, apple, orange, pear,

    pineapple and banana). Point to the rst person in the

    plenary line o chairs and say apple, move to the secondand say orange and so on until you are back to appleor the sixth participant. Repeat until everyone has an

    assigned ruit.

    Tell the participants to swap places with others with the

    same assigned ruits when you call out the name o their

    ruit. Shout apples and everyone must run to swap places

    with someone else. Repeat or all ruits randomly. The

    participants can then orm ve groups based on the

    ve ruits.

    Time

    10 minutes

    Instructions

    1. Explain to participants that you will give a presentation outlining some tips or developing

    evaluation questions and some examples.

    2. Ask participants i they have any questions. Discuss as necessary.

    ONE-DAy wORkSHOP

  • 8/2/2019 Measuring up: HIV-related Advocacy Evaluation Training for Civil Society Organisations (facilitator's guide)

    27/38

    25MEASURING UP \\A GUIDE FOR FACILITATORS

    DAY TWO

    SESSION 3.5 Deciding on an approach to measurement

    Session inormation

    Inormation or this session canbe ound on pages 20 and 21 o

    the learners guide.

    Facilitators notes

    You may want to use or adapt

    Measuring up workshop

    PowerPoint 8 Module 3:

    Deciding on an approach

    to measurementor thepresentations during this

    session.

    Time25 minutes

    Instructions

    1.Introduce the name o the session and its place in the

    evaluation design ramework fip chart.

    2. Explain to participants that you will give a presentation

    outlining ve practical and strategic approaches to

    measurement put orward by Organizational Research

    Services or advocacy evaluation. Give the presentation.

    3. Ask participants i they have any questions about the ve

    practical and strategic approaches to measurement.

    Time

    10 minutes

    Instructions

    1. Explain to participants that you will give a presentation outlining some dierent approaches to

    measuring.

    2. Ask participants i they have any questions. Discuss as necessary.

    ONE-DAy wORkSHOP

  • 8/2/2019 Measuring up: HIV-related Advocacy Evaluation Training for Civil Society Organisations (facilitator's guide)

    28/38

    26MEASURING UP \\A GUIDE FOR FACILITATORS

    DAY TWO

    SESSION 3.6 Selecting indicators

    Session inormation

    Inormation or this session canbe ound on pages 22 and 25 o

    the learners guide.

    Materials

    For each pair o participants:

    l an Indicator selection

    worksheet and copy o the

    Example indicators rom the

    Measuring up: workshop

    supplementary acilitation

    material

    l have some extra copies o theworksheet available in caseparticipants ask or a clean

    sheet

    Time

    1 hour

    Facilitators notes

    l You may want to use or adapt Measuring up workshop

    PowerPoint 9 Module 3: Selecting indicators or the

    presentations during this session. You have a maximum o 30minutes.

    l Prepare resources or each pair o participants in advance.

    Instructions

    1. Introduce the name o the session and its place in the

    evaluation design ramework fip chart.

    2. Remind participants that they will need their learners guide

    during the presentation. Give the presentation.

    3. Explain to participants that a quick learning activity will ollowto help amiliarise them with the indicators in the learners

    guide and give them practice in writing tailored indicators.

    The activity will be done in buzz pairs (people rom dierent

    organisations). Hand each pair an indicator selection

    worksheet and a copy o the example indicators rom the

    Measuring up: workshop supplementary acilitation material.

    Ask the buzz pairs to complete the table with examples o

    output and outcome indicators. Ask them not to cheat by

    looking in the learners guide! Tell participants they have 20

    minutes to work in pairs.

    4. Circulate around the pairs to help and also to identiy goodexamples to share in plenary. Point out these good examples

    to the pairs so that you can draw on them in the plenary.

    5. Ater 20 minutes ask everyone to stop writing indicators

    and share some examples. Encourage those with the good

    examples you identied to contribute. Look at the examples in

    table 5 on page 25 o the learners guide.

    6. Hold a brie discussion or ask or any questions.

    Time

    20 minutes

    Instructions

    1. Explain to participants that you will give a presentation on selecting indicators.

    2. Ask participants i they have any questions. Discuss as necessary.

    ONE-DAy wORkSHOP

  • 8/2/2019 Measuring up: HIV-related Advocacy Evaluation Training for Civil Society Organisations (facilitator's guide)

    29/38

    27MEASURING UP \\A GUIDE FOR FACILITATORS

    DAY THREE

    DAy THREE Recap o day two

    Facilitators notes

    It is useul to use a combinationo the workshop timetable

    and fip charts rom output or

    plenary brainstorms to review

    key points rom the previous

    day.

    Time

    30 minutes

    Instructions

    1. Facilitate an energiser.2. Provide a recap o the previous days sessions.

    SESSION 3.7Identiying and choosing data collection methods relevant

    to advocacy evaluation

    Session inormation

    Inormation or this session can

    be ound on pages 26 to 36 o

    the learners guide.

    Materialsl PowerPoint 10

    l Copies o the contents

    rom A handbook o data

    collection tools: a companion

    to A guide to measuringadvocacy and policyrom

    the Measuring up: workshop

    supplementary acilitation

    material. One copy or each

    participant and the acilitator.

    Time2 hours (not including the

    rereshments break)

    Facilitators notes

    l I participants were not already aware o the seven data

    collection methods beore the workshop, they may have read

    the learners guide overnight and now be amiliar with theseapproaches.

    l You may want to use or adapt Measuring up workshop

    PowerPoint 10 Module 3: Identiying and choosing data

    collection methods relevant to advocacy evaluation or the

    presentations during this session.

    l Keep an eye on the time and make sure that you stop or the

    rereshments break.

    l Participants are likely to ask where to access more inormation

    about these methods. There is limited inormation available

    at the moment, but they should keep searching the Internet

    as this situation may change quickly. Recommend that

    participants who wish to track the work o key authors

    subscribe to The evaluation exchange periodical, which is reeo charge (google The evaluation exchange). This is a orum

    or sharing new and innovative data collection methods,

    including or advocacy and policy evaluation.

    Instructions

    1. Introduce the name o the session and its place in the

    evaluation design ramework fip chart. Explain that the

    session will be divided by a rereshments break.

    2. Explain that data collection methods are the processes or

    approaches used or collecting data to measure our indicators.

    3. Ask participants: Can you identiy some common data

    collection methods? Lets brainstorm them.

  • 8/2/2019 Measuring up: HIV-related Advocacy Evaluation Training for Civil Society Organisations (facilitator's guide)

    30/38

    28MEASURING UP \\A GUIDE FOR FACILITATORS

    DAY THREE

    4. Ask participants: Do you know o any new data collectionmethods designed or advocacy evaluation?

    5. Explain to participants that you will present data collectionmethods or advocacy work. This will include an introduction

    to publications that include relevant tools, an overview o

    common data collection methods and an explanation o seven

    data collection methods useul or advocacy evaluation. Tell

    them that all the inormation or this presentation can beound in the learners guide in section 3.7, on pages 26 to 36,

    except or the contents o A handbook o data collection tools:

    a companion to A guide to measuring advocacy and policy,

    which will be handed out now. Explain that we will be looking

    at the seven data collection methods in more detail in group

    work, so participants should write down any questions they

    have during the presentation. Give the presentation.

    6. Ater the presentation, acilitate a discussion with the

    participants about the seven evaluation methods based on the

    ollowing kinds o questions:

    l Has anyone in your group used this data collection

    method beore?l What do you think are some o the key strengths o

    this method?(Dont orget to point out any particular

    strengths or evaluating HIV-related advocacy)

    l What aspects o this method may be challenging?(Dont

    orget to point out any particular challenges or evaluating

    HIV-related advocacy.)

    Bellwether methodology can be used at the

    national level in southern Arican countries

    International HIV/AIDS Alliance partners is

    Southern Arica were interested in whether

    the Bellwether methodology is appropriate

    or evaluating advocacy or treatment access.

    The answer is it depends on the context,

    method and indicators. For instance, i there was

    political and health system support or access

    and advocacy was directed around pricing and

    suppliers, then the best indicator would simply

    be the number o people enrolled in treatment,

    which could be veried rom clinic data. However,

    i there was an aordable treatment supply, butproblems o stock-outs and lack o commitment

    rom the health system or health minister toaddress this, the advocacy intervention might be

    to try to raise awareness o the consequences

    o stock-outs among other infuential people

    who could persuade the minister to act. Then

    bellwether could be a helpul evaluation

    technique. Measuring up contributor.

    The intense period debrie method can be used at

    the national level in Kenya

    See the example provided or the partnership

    analysis tool (learners guide, page 33).

    KANCO and its partners could ask the ollowing

    intensive period debrie questions:

    l What events triggered the need or thisintense period o advocacy?

    l How would you describe the political context

    o this period concerning the HIV and AIDS

    Prevention and Control Act?

    l What was the general public mood concerning

    the HIV and AIDS Prevention and Control Act

    when the advocacy action began?

    l What did KANCO do well? What did not work

    well and could have been improved?

    l What aspects o the advocacy partnership

    worked well and what did not? How couldsuch an advocacy partnership be improved in

    the uture?

    l Which advocacy strategies employed were the

    most and least eective in achieving change?

    l As a result o this intense period o advocacy

    activity, what new opportunities have arisen

    or the legislative campaign or the HIV Act

    going orward?

    l What insight will you take away rom the

    experience that could inorm advocacy

    strategy development in the uture?

    * Not included in the learners guide or PowerPoint presentations

    Time

    40 minutes

    Instructions

    1. Ask participants: Can you

    identiy some common data

    collection methods? Lets

    brainstorm them.

    2. Ask participants: Do you know

    o any new data collection

    methods designed or advocacy

    evaluation?

    3. Explain to participants that

    you will give a presentation

    outlining seven data collection

    methods or advocacyevaluation (see page 27 in the

    learners guide).

    4. Ask participants i they have

    any questions. Discuss asnecessary.

    ONE-DAy wORkSHOP

    Additional examples o the use o specic methods*

  • 8/2/2019 Measuring up: HIV-related Advocacy Evaluation Training for Civil Society Organisations (facilitator's guide)

    31/38

    29MEASURING UP \\A GUIDE FOR FACILITATORS

    DAY THREE

    MODULE 4

    SESSION 4.1

    Putting it all into action introduction to Module 4

    Consolidating learning on how to evaluate HIV-related

    advocacy

    Time

    5 minutes

    Instructions

    1. Introduce the name o the session and its place in the

    workshop timetable.

    2. Give a brie introduction to this module, including the title and

    objectives.

    Objectives or Module 4

    l To practise using the seven key components

    o the advocacy evaluation design ramework

    or decision-making on how to evaluate HIV-

    related advocacy.

    l To refect on the key messages and learning

    points rom the workshop to share withcolleagues.

    Session inormation

    Inormation or this session

    can be ound on page 37 o the

    learners guide.

    Materials

    l Participants previous group

    work activity outputs rom

    sessions 3.2 and 3.3

    l A copy or each participant

    o the group work activity

    instructions on page 29

    l Flip chart paper, fip chart

    pens and tape (Stickie Stu or

    Blu-Tak, etc.)

    Time3 hours (not including the lunchbreak)

    Facilitators notes

    l Make sure that you read through the example theory ochange careully beore acilitating this session.

    l Keep an eye on the time and make sure that you stop or the

    lunch break

    l Prepare resources or each group and ensure there is a space

    to work, with a table and correct number o chairs (or foor, i

    appropriate). Make sure you do this in advance.

    l Circulate around the groups to help them get started with the

    activity and then monitor their progress periodically to help

    them keep time and address any questions they might have.

    l Identiy which group you would like to present their work and

    warn them in advance.

    Instructions

    1. Introduce the name o the session and its place in the

    workshop timetable.

    2. Explain to participants that they will work in their organisation

    group using their previous work on developing a theory

    o change to help them practise designing an advocacy

    evaluation in line with the last our components o the

    advocacy evaluation design ramework. (Use the advocacyevaluation ramework fip chart to point to these last

    our components evaluation questions, approaches to

    measurement, indicator selection and selection o data

    collection methods.)

  • 8/2/2019 Measuring up: HIV-related Advocacy Evaluation Training for Civil Society Organisations (facilitator's guide)

    32/38

    30MEASURING UP \\A GUIDE FOR FACILITATORS

    DAY THREE

    3. Introduce the group work instructions to participants using aprepared fip chart.

    4. Hand out a copy o the activity instructions.

    5. Ater two hours ask participants to return to plenary. Facilitatea 45-minute discussion about the activity by asking one group

    to present its work. Invite other participants to comment on

    their experience o the activity, including what they learned

    rom the process.

    Group activity instructions:

    1. You have two hours to complete this activity.

    2. Answer the ollowing questions to help you to practise designing an

    advocacy evaluation related to your work:

    l What assumptions have been made in the design o your advocacy

    eort? (Learners guide, Module 3, section 3.2, pages 12 to 17)

    l What evaluation questions would be inormative or this advocacy

    work? (Learners guide, Module 3, section 3.4. page 19)

    l What approach(es) to measurement do you think would be appropriate

    or this advocacy work? (Learners guide, Module 3, section 5, pages 20

    and 21)

    l What would you prioritise or this evaluation and what evaluation

    questions would be useul or the end users? (Learners guide, module

    3, sections 3.3 and 3.4 pages 18 and 19)

    l What key indicators will allow you to measure the results o thisadvocacy work? (Learners guide, Module 3, section 3.5, pages 22 to 25)

    l What are some sources and/or means o data collection or these

    indicators? (Learners guide, Module 3, section 3.7, pages 26 to 36.)

    3. You may be asked to present your group work, so write the answers to your

    questions on fip chart paper.

  • 8/2/2019 Measuring up: HIV-related Advocacy Evaluation Training for Civil Society Organisations (facilitator's guide)

    33/38

    31MEASURING UP \\A GUIDE FOR FACILITATORS

    DAY THREE

    Time

    2 hours 30 minutes

    Facilitators notes

    l Make sure that you read through the example theory o change careully beore acilitating this

    session.

    l You could use the Lieboats energiser, see page 34, to create the groups o our people.

    l Each group will need the ollowing resources or the group work (make sure you prepare these in

    advance):

    l one copy or each participant (our copies per group) o the group work activity instructions,

    case study and example theory o change printed o rom the Measuring up: workshop

    supplementary acilitation material

    l fip chart paper, fip chart pens and tape (Stickie Stu, Blu-Tak, etc.)

    l a space to work, with a table and correct number o chairs (or foor, i appropriate).

    l Circulate around the groups to help them get started with the activity and then monitor their

    progress periodically to help them keep time and address any questions they might have. Identiy

    which group you would like to present its work and warn the group participants in advance.

    Instructions

    1. Introduce the names o sessions 4 and 4.1 and their place in the workshop timetable.

    2. Introduce the objectives or Module 4.

    3. Explain to participants that they will work in groups o approximately our people to practiseapplying the advocacy evaluation design ramework to an example.

    4. Review the case study example and related theory o change or improved access to harm

    reduction services or injecting drug users included in the Measuring up: workshop supplementary

    acilitation material.

    5. Introduce the group work instructions to participants using a prepared fip chart.

    6. Hand a copy o the activity instructions, case study and example theory o change to each

    participant.

    7. Use an energiser to create the groups o our.

    Group work instructions

    a) You have 1 hour 30 minutes to complete this activity.

    b) Read through the case study and example theory o change as a group. Do not debate the

    design o the advocacy eort. The evaluation will be conducted ater three years o this ve-

    year unded advocacy initiative. The primary users o the evaluation will be the advocates

    themselves and donors, and the secondary users will be other advocates wishing to advocate

    or harm reduction. The evaluation will be used to (i) demonstrate the eectiveness o the

    advocacy coalition, which is unded through a lead partner; and (ii) provide an opportunity or

    the advocates to learn more about the eectiveness o one or more approaches or improved

    strategy development and evidence-based learning or uture advocacy work.

    ONE-DAy wORkSHOP

  • 8/2/2019 Measuring up: HIV-related Advocacy Evaluation Training for Civil Society Organisations (facilitator's guide)

    34/38

    32MEASURING UP \\A GUIDE FOR FACILITATORS

    DAY THREE

    ONE-DAy wORkSHOP (CONTINUED)

    c) Answer the ollowing questions to help you to practise applying the inormation covered

    during the workshop:

    l What assumptions have been made in the design o the advocacy eort?

    (Learners guide, Module 3, section 2, pages 12 to 17)l What elements o the advocacy work would you prioritise or evaluation?

    (Learners guide, Module 3, section 3.3, page 18)

    l What evaluation questions would be inormative or this advocacy work?

    (Learners guide, Module 3, section 3.4, page 19)

    l What approach to measurement do you think would be appropriate or this advocacywork? (Learners guide, Module 3, section 5, pages 20 and 21)

    l What key indicators will allow you to measure results o this advocacy work?(Learners guide, Module 3, section 3.6, pages 22 to 25)

    l What are some sources and/or means o data collection or these indicators?(Learners guide, Module 3, section 3.7, pages 26 to 36)

    d) You may be asked to present your group work, so write the answers to your questions on fipchart paper.

    Please note that there are no right or wrong answers. We ask that you think back to what has

    been presented so ar, as well as to your experience on the ground.

    8. Ater 1 hour 30 minutes ask participants to return to plenary. Facilitate a 40-minute discussion

    about the activity by asking one group to present their work and inviting the other participants

    to comment and suggest amendments, additions, etc. Make sure that you and the participants

    critique the ideas suciently in order to create a good example.

    Activity example output: using the people who use drugs case study rom the supplementary

    acilitation materials to consolidate learning on how to evaluate HIV-related advocacy

    What assumptions have been made in the design

    o the advocacy eort?

    l The government ocials and police authority

    representatives that are accessible or

    targeting with advocacy messages and

    training are those who have enough infuence

    to achieve change.

    l Education concerning the public health

    benets o harm reduction services will allow

    individuals rom the government, police and

    public to overcome their prejudices and thebarriers/obstacles put in place o access to

    harm reduction services.

    What elements o the advocacy work would you

    prioritise or evaluation?

    l Were the right government ocials and

    police authority representatives targeted with

    advocacy messages and training?

    l Eective joint actions and aligned messagingo the coalition.

    l Changes in political will/commitment to adopt

    policies that are sensitive to the needs o

    people who use drugs.

    l Changes in the position o harm reduction on

    the policy agenda or HIV prevention.

  • 8/2/2019 Measuring up: HIV-related Advocacy Evaluation Training for Civil Society Organisations (facilitator's guide)

    35/38

    33MEASURING UP \\A GUIDE FOR FACILITATORS

    DAY THREE

    Activity example output: (continued)

    What evaluation questions would be inormative

    or this advocacy work?

    l Were the government ocials/police

    authority representatives invited to round-

    table discussions/training and targeted with

    coalition messages well selected to aect the

    necessary change?

    l How eective has the coalition been in its

    message development and targeting or

    government ocials and the police authority?

    l Has the coalition been able to contribute to

    increased political will to support improvedaccess to harm reduction services and

    policies or legislation that supports this? I so,

    how? I not, why not?

    l

    Has the coalition been able to contribute toimproved policies or legislation that supports

    this improved access to harm reduction

    services? I so, how? I not, why not?

    l To what extent has the coalitions media

    campaign contributed to progress to improve

    access to harm reduction services?

    What approach to measurement do you think

    would be appropriate or this advocacy work?

    Approach A: identication and measurement o

    core outcome areas related to social or policy

    change.

    What key indicators will allow you to measurethe results o this advocacy work?

    l Number o civil society organisations

    targeted to join the advocacy coalition.

    l Percentage o civil society organisations

    targeted to join the advocacy coalition that

    have joined the coalition.

    l Number and type o organisations that have

    joined the coalition.

    l Number o incidences that collation key

    messages are used avourably in the media.

    l Number o incidences that collation

    key messages are used avourably by

    government ocials.

    l Number o training events/round tables

    or government ocials/police authorities

    provided on the issue.

    l Number o government ocials/police

    authority representatives attending round

    table discussions/training events on the issue.

    l Improved political will o government ocials

    to improve access and reduce