communication for development (c4d) learning...

35
Communication for Development (C4D) Learning Course Module 3 of 3

Upload: vantuong

Post on 12-Mar-2018

226 views

Category:

Documents


4 download

TRANSCRIPT

Page 1: Communication for Development (C4D) Learning Courseprofessorcarlson.net/c4dcourse/orientation/orient_unit1/UNICEFC4D... · a communication strategy/initiative to be broader in scope

Communication for Development (C4D) Learning CourseModule 3 of 3

Page 2: Communication for Development (C4D) Learning Courseprofessorcarlson.net/c4dcourse/orientation/orient_unit1/UNICEFC4D... · a communication strategy/initiative to be broader in scope

This workbook supplements the online learning materials. It contains the text of the module, but without the readings. The complete version of the module and all accompanying materials are on the course website and the DVD and CD sent to you.

In developing this course, the Communication and Development Studies Program and the Center for International Studies at Ohio University have made every effort to trace copyright holders and adhere to educational fair usage guidelines. We regret omissions and welcome additions and amendments for inclusion in future revisions.

© Ohio University and UNICEF 2013. This material is not licensed for resale or copying without agreement

Communication for Development (C4D)Learning Course

MODULE 3 OF 3

Module 1: Values, Principles, and Concepts in C4D 01

Page 3: Communication for Development (C4D) Learning Courseprofessorcarlson.net/c4dcourse/orientation/orient_unit1/UNICEFC4D... · a communication strategy/initiative to be broader in scope

Module ThreeTable of Contents

1. Introduction to Module 3 73

2. learning objectives 74

3. unit Structure 76

4. unit 1: logic Models 77a. Introduction 77b. learning objectives 77c. Background 78d. discussion 78e. readings and resources 79

5. unit 2: Communication Strategy design 82a. Introduction 82b. learning objectives 82c. discussion 83d. readings and resources 84

6. unit 3: research design and Techniques 86a. Introduction 86b. learning objectives 86c. Background 87d. discussion 89e. readings and resources 90

7. unit 4: Programme evaluation Methodologies 94a. Introduction 94b. learning objectives 94c. Background 94d. discussion 96e. readings and resources 96

04 Table of Contents

Page 4: Communication for Development (C4D) Learning Courseprofessorcarlson.net/c4dcourse/orientation/orient_unit1/UNICEFC4D... · a communication strategy/initiative to be broader in scope

70 Module 3: Research, Methods, and Practice in C4D

Page 5: Communication for Development (C4D) Learning Courseprofessorcarlson.net/c4dcourse/orientation/orient_unit1/UNICEFC4D... · a communication strategy/initiative to be broader in scope

Research, Methods, and Practice in C4D

Communication for Development: A UNICEF-Ohio University Learning Course

MODULE 3:

Page 6: Communication for Development (C4D) Learning Courseprofessorcarlson.net/c4dcourse/orientation/orient_unit1/UNICEFC4D... · a communication strategy/initiative to be broader in scope

72 Module 3: Research, Methods, and Practice in C4D

Page 7: Communication for Development (C4D) Learning Courseprofessorcarlson.net/c4dcourse/orientation/orient_unit1/UNICEFC4D... · a communication strategy/initiative to be broader in scope

You’ve now worked your way through values, principles and concepts in C4D, and explored how individual, interpersonal and community-level change theories guide C4D design and implementation. Now it’s time to bring together what you’ve learned and start planning C4D strategies, interventions, and evaluations. Module 3 covers the methods and techniques you’ll need for planning, implementing, and evaluating C4D programmes and will help prepare you for the face-to-face workshop.

INTRODUCTION TO MODULE 3: Research, Methods, and Practice in C4D

73Module 3: Research, Methods, and Practice in C4D

Page 8: Communication for Development (C4D) Learning Courseprofessorcarlson.net/c4dcourse/orientation/orient_unit1/UNICEFC4D... · a communication strategy/initiative to be broader in scope

Learning Objectives

Module Three focuses on the application and integration of principles, concepts, and behaviour and social change theories, to develop evidence-informed C4D strategies and initiatives. You will be challenged to apply the learning from this and the previous two modules.

At the end of this module you will be able to:

• Describe the different components of a logic/planning model• Describe strategies for communicating with hard-to-reach groups• Describe different research techniques • Differentiate between formative, monitoring (process evaluation), and impact

evaluation research processes in C4D

74 Module 3: Research, Methods, and Practice in C4D

Page 9: Communication for Development (C4D) Learning Courseprofessorcarlson.net/c4dcourse/orientation/orient_unit1/UNICEFC4D... · a communication strategy/initiative to be broader in scope

In designing, implementing, and evaluating C4D strategies it is essential to understand the linkages between theory, C4D initiatives, and research. A theory-based approach to C4D planning and evaluation allows you to identify critical areas (for example, changes in knowledge, attitudes, practices, and norms) and measure the effectiveness of initiatives in these areas using appropriate evaluation techniques. The material in this module will help you understand these linkages and create more effective communication strategies.

One of the most helpful ways to plan a C4D intervention is to use a logic model (Unit 1). A logic model is simply a diagram, a flow chart that links C4D activities and outputs with expected outcomes. It doesn’t need to be great art; indeed, some excellent logic models have been drafted on paper napkins in restaurants after a long day at the office. It’s a way to break down your written strategy and see how all the pieces fit together. The logic model includes programme goals and measurable objectives (we’ll be going over the difference between them), and shows how research, design, implementation and monitoring, and evaluation are connected.

Unit 2 is about designing communication strategies. This will help you develop strategies and initiatives that are based on evidence (theory and research), reflect UNICEF’s mission, goals, and values and recognize local social, cultural, and political contexts. We’ll be using examples from mass media, community, and interpersonal communication/counselling.

Unit 3 provides an overview of research methods and techniques. UNICEF is committed to participatory evaluation techniques—those that involve families and children—so that’s our starting point. But we know you’ll also need to use (and sometimes commission) other types of research. We’ll look at the advantages and disadvantages of quantitative and qualitative methodologies and pose the question: how do you figure out what’s good (a solid methodology and analysis) and what isn’t? We introduce you to the concepts of reliability and validity to help you make those judgments.

In Unit 4, we take you through the process of monitoring and evaluating your initiative. We’ll look at the research you do when planning a C4D initiative, the evaluations you do when programmes are being implemented (monitoring or process evaluation), and after they’re completed (impact evaluation).

Note: In this Module you will notice that we use both intervention and strategy to describe communication-related work. For the purposes of this course, you may consider a communication strategy/initiative to be broader in scope than a communication intervention.

75Module 3: Research, Methods, and Practice in C4D

Page 10: Communication for Development (C4D) Learning Courseprofessorcarlson.net/c4dcourse/orientation/orient_unit1/UNICEFC4D... · a communication strategy/initiative to be broader in scope

An outline of the unit, indicating how the topics covered relate to C4D.

Key concepts and the broader C4D and UNICEF context.

Required and optional readings, resources and tools to help you complete the unit.

Questions for discussion about the issues in the unit.

Introduction

Background

Discussion

Readings and Resources

Each module consists of four units. Each unit has a common structure with five elements, identified by images on the website and in the workbooks.

What you will know and/or be able to do after completing the unit.

Learning Objectives

1

2

3

4

5

Unit Structure

A question or an optional response

Additional information about a topic

More Navigational Images

Video materials

76 Module 3: Research, Methods, and Practice in C4D

Page 11: Communication for Development (C4D) Learning Courseprofessorcarlson.net/c4dcourse/orientation/orient_unit1/UNICEFC4D... · a communication strategy/initiative to be broader in scope

In this unit, we will focus on key elements associated with researching, designing, implementing, monitoring, and evaluating C4D strategies. This includes a broad overview of logic models. A logic model is a visual representation linking programme and C4D activities and outputs with expected outcomes (long and short term). It serves as a road map to research, design, implement, monitor, and evaluate initiatives.

The unit also includes information about the process of setting goals and objectives for a programme/C4D strategy. In order to measure effectiveness, it is important that our objectives follow the SMART criteria:

S = Specific M = Measureable A = Achievable R = Realistic and T = Time Bound.

In addition, from the participatory point of view it is also important that qualitative objectives be SPICED:

S = Subjective P = ParticipatoryI = Interpreted and CommunicableC = Cross-checked and ComparedE = EmpoweringD = Diverse and Disaggregated.

Based on the readings and discussion, by the end of this unit, you will be able to:• Differentiate between programme

goals that include C4D (large scale, long term achievements related to social change) and objectives (short to medium term achievements).

• Differentiate between C4D activities, outputs, and outcomes

MODULE 3 / UNIT 1:Logic Models

LEARNING OBJECTIVESINTRODUCTION

77Module 3: Research, Methods, and Practice in C4D

Page 12: Communication for Development (C4D) Learning Courseprofessorcarlson.net/c4dcourse/orientation/orient_unit1/UNICEFC4D... · a communication strategy/initiative to be broader in scope

There are many excellent logic and planning models currently in use by organizations across the world. The COMBI model (Communication-for- Behavioural-Impact) is currently being used by the World Health Organization (WHO). Johns Hopkins University’s Center for Communication Programs uses a planning tool called SCOPE (Strategic Communication Planning and Evaluation), which is the interactive computer-based version of a 5-step planning tool called the “P-Process”. UNICEF has developed a tool called the ACADA model (Assessment, Communication Analysis, Design, and Action).

In this unit we present the basic principles featured in each of the models rather than focusing on one particular model. If you are interested in learning more about specific models we invite you to view the summaries listed in the Optional Reading section.

Equally important in the development of any C4D strategy and plan is active collaboration with relevant specialists, local partners, government counterparts and other UN and NGO partners. Ultimately, strategic decisions regarding the resourcing, design, and implementation of a C4D initiative are always made consultatively.

As with each of the units in this module, this introduction to logic and planning models is based on the analysis and use of evidence and demonstrated past performance in the development, implementation and measurement of new strategies.

Before beginning the readings on logic models and planning models, discuss any past experience (positive or negative) with logic models and/or planning models.

1. Briefly describe the process or outcome of using the logic or planning model and state whether or not you were satisfied with the tool, and why/why not?

2. Create a brief logic model consisting of:

Your Planned Work (Inputs, Activities) + Your Intended Results (Outputs, Outcomes, Impact)

For this discussion, you have the option of using the data in the case study below. In the Readings and Resources section, you will find tools and templates related to logic models. • Case Study 4 (Tokrali) encouraging

the delay of sexual debut among youth.

BACKGROUND DISCUSSION

78 Module 3: Research, Methods, and Practice in C4D

Page 13: Communication for Development (C4D) Learning Courseprofessorcarlson.net/c4dcourse/orientation/orient_unit1/UNICEFC4D... · a communication strategy/initiative to be broader in scope

READINGS AND RESOURCES

[Note: Readings and resources are available on the CD or DVD accompanying this workbook, along with the full citations for all supplied readings.]

REQUIRED READINGSCenters for Disease Control and Prevention (2009, January). Writing SMART Objectives. Evaluation Briefs No.3b. [2 p.]

UNICEF (n.d.). What is a programme logic model? UNICEF M & E Training Resource. [7 p.]

MDF (2005). MDF Tool - Indicators. [10 p.]

Ohio University Communication and Development Studies (2011). Goals and objectives [powerpoint presentation]. [17 slides].

UNICEF Bangladesh (2008). Writing a Communication Strategy for Development Programs. [Chapter 1 – 10 p.]

World Health Organization (2003). Mobilizing for Action: Communication for Behavioural Impact. [4 p.]

REQUIRED VIEWINGVideo 1: Logic ModelsNote: This video features a training module on Logic Models, which was originally interactive. This recorded version no longer allows you to click on the presentation.

OPTIONAL TOOLSTools are provided for your reference, and for you to use in your subsequent work, when appropriate.

Tools for Logic Models Logic Model Template

Tools for Goals and Objectives Writing Objectives Template for Case Studies pre-activity Writing Objectives Activity Writing Objectives Template for Case Studies post-activity

79Module 3: Research, Methods, and Practice in C4D

Page 14: Communication for Development (C4D) Learning Courseprofessorcarlson.net/c4dcourse/orientation/orient_unit1/UNICEFC4D... · a communication strategy/initiative to be broader in scope

OPTIONAL READINGSCommunication for Development, UNICEF NY., (n.d.). ACADA model summary. Retrieved from http://www.unicef.org/cbsc/index_42336.html. [2 p.]

Innovation Network (n.d.). Logic Model Workbook. Retrieved from www.innonet.org. [30 p.]

Johns Hopkins Bloomberg School of Public Health (2009). The research process. Retrieved from http://ocw.jhsph.edu/courses/entertainmenteducation/PDFs/Research_Process.pdf. [12 p.]

Minnesota Department of Health (2010). Program goals and objectives [web page]. Retrieved from http://www.health.state.mn.us/divs/hpcd/chp/hpkit/text/team_goals.htm. [2 p.]

W.K. Kellogg Foundation (2004). Logic Model Development Guide. Retrieved from http://www.wkkf.org/~/media/6E35F79692704AA0ADCC8C3017200208.ashx. [71 p.]

80 Module 3: Research, Methods, and Practice in C4D

Page 15: Communication for Development (C4D) Learning Courseprofessorcarlson.net/c4dcourse/orientation/orient_unit1/UNICEFC4D... · a communication strategy/initiative to be broader in scope

Notes

81Module 3: Research, Methods, and Practice in C4D

Page 16: Communication for Development (C4D) Learning Courseprofessorcarlson.net/c4dcourse/orientation/orient_unit1/UNICEFC4D... · a communication strategy/initiative to be broader in scope

In this unit we will discuss the key elements that go into the design and implementation of a C4D strategy. Examples from mass media, community, and interpersonal communication/ counselling will be utilized to illustrate design issues. You will be introduced to a step-by-step approach for planning a communication strategy. Recall that the optional readings section of Unit 1 has links to several specific planning models (COMBI, SCOPE/P-Process, ACADA, etc.). This unit will provide you with step-by-step guidance on key elements to consider in the development of a strategy. These steps can then be used as a basis to facilitate a process of collaborative strategy development in your country, as you work with government counterparts, UN and NGO partners, and grass-roots organisations.

Remember that Module 1, Unit 4 also contains information on Communication Strategy Design.

Based on the readings and discussion, by the end of this unit, you will be able to:• Identify relevant communication

processes, practices and channels within the community, country or context in which you work.

• Describe strategies for fostering participation and/or addressing the needs of “hard to reach” groups while respecting local socio-cultural, economic and political contexts and issues.

MODULE 3 / UNIT 2:Communication Strategy Design

LEARNING OBJECTIVESINTRODUCTION

82 Module 3: Research, Methods, and Practice in C4D

Page 17: Communication for Development (C4D) Learning Courseprofessorcarlson.net/c4dcourse/orientation/orient_unit1/UNICEFC4D... · a communication strategy/initiative to be broader in scope

Most logic and planning models begin with some form of analysis (situation analysis, “SWOT” analysis — Strengths, Weaknesses, Opportunities, and Threats, formative research etc.)

DISCUSSION

Discuss these questions:

1. What are the most common forms of communication used in your office for reaching out to communities?

2. What strategies have you observed — or would you recommend — for addressing the needs of “hard to reach” groups when developing a communication strategy?

3. To communicate with “hard to reach” groups does it take additional resources or is it simply a question of being more sensitive and inclusive at the strategy design phase?

If you are not currently in a field staff position, use your immediate context — or a UNICEF field context you are familiar with — to answer the second question.

83Module 3: Research, Methods, and Practice in C4D

Page 18: Communication for Development (C4D) Learning Courseprofessorcarlson.net/c4dcourse/orientation/orient_unit1/UNICEFC4D... · a communication strategy/initiative to be broader in scope

[Note: Readings and resources are available on the CD or DVD accompanying this workbook, along with the full citations for all supplied readings.]

REQUIRED READINGS

Tufte, T. & Mefalopulos, P. (2009) The four phases of the communication program cycle. In Participatory Communication. A Practical Guide. World Bank Working Paper # 170. (pp. 20 – 36). The World Bank. Copenhagen and Washington DC. [16 p.]

Johns Hopkins Bloomberg School of Public Health. (2008). Communication for Better Health. Population Reports. S (56). [28 p.]

OPTIONAL READINGS

McKee, N. et. al. 2008. “Involving people, evolving behavior: The UNICEF experience.” In Servaes, J. (Ed.) Communication for Development and Social Change. UNESCO/Sage. [24 p.]

[Note: These two readings were provided in Unit 1].

UNICEF Bangladesh (2008). Writing a Communication Strategy for Development Programs. [Chapter 1 – 10 p.]

World Health Organization (2003). Mobilizing for Action: Communication for Behavioural Impact. [4 p.]

READINGS AND RESOURCES

84 Module 3: Research, Methods, and Practice in C4D

Page 19: Communication for Development (C4D) Learning Courseprofessorcarlson.net/c4dcourse/orientation/orient_unit1/UNICEFC4D... · a communication strategy/initiative to be broader in scope

Notes

85Module 3: Research, Methods, and Practice in C4D

Page 20: Communication for Development (C4D) Learning Courseprofessorcarlson.net/c4dcourse/orientation/orient_unit1/UNICEFC4D... · a communication strategy/initiative to be broader in scope

In this unit we provide an overview of various research designs and techniques. The readings cover general information on different qualitative and quantitative methodologies, including the concepts of reliability and validity in research, and describe how both qualitative and quantitative research can be more inclusive and participatory.

As a UNICEF officer, you may be tasked with employing consultants or providing vital inputs to conduct research related to a communication programme. For this reason, it is necessary for you to understand the broad concepts of reliability and validity, while maintaining UNICEF’s commitment to participatory research techniques that allow communities to contribute to the development process.

Based on the readings and discussion, by the end of this unit, you will be able to:

• Discuss the importance of research in designing and implementing C4D programmes.

• Identify the strengths and limitations of different qualitative and quantitative methodologies commonly used in theory-driven, social research as is relevant to the practice of C4D.

• Describe how research can be designed to invite participation.

LEARNING OBJECTIVESINTRODUCTION

MODULE 3 / UNIT 3:Research Design and Techniques

86 Module 3: Research, Methods, and Practice in C4D

Page 21: Communication for Development (C4D) Learning Courseprofessorcarlson.net/c4dcourse/orientation/orient_unit1/UNICEFC4D... · a communication strategy/initiative to be broader in scope

BACKGROUND

The focus of the readings in this unit is on research design and techniques that can be applied during the entire programme cycle. In unit 4 you will learn more about specific monitoring and evaluation techniques, and you will note that there is some overlap in the research techniques described.

Since this module is limited in time, it will not be possible to cover ALL of the potential research methods available to communication and social science researchers. We encourage you to engage with the readings, including optional readings, as much as you can, and to bring your interests in a particular method or technique to the face-to-face workshop, where we will explore research techniques in further detail.

This table, on the next page, provides a comparison of the range, strengths and limitations of qualitative and quantitative research.

Research methods can be combined, sequenced and made participatory through design and planning. Research specialist Uwe Flick suggests that research methods, when combined, can complement the weaknesses and blind spots of a single method.1

Qualitative interviews can help researchers understand issues in a given community, and this data can inform the design of a quantitative survey that tests how many people, in which areas of the community, experience specific issues. The inverse sequence can also be fruitful: A quantitative survey that identifies a set of issues (or attitudes, or barriers, etc.)

frequently experienced in a community can be followed with a qualitative study that seeks richer details and explanations to complement survey data.

Participation can be invited at any point in the research process. Both interview and survey methods can be participatory, for example, if young people or other community members are trained to conduct interviews or surveys, or if they are invited to a community data analysis session. In the Readings section, the article on the Zimeselini peer research project in South Africa gives an excellent description on the benefits, and challenges, of participatory forms of research.

1Flick, U. (2009), An introduction to qualitative

research, Sage, London.

87Module 3: Research, Methods, and Practice in C4D

Page 22: Communication for Development (C4D) Learning Courseprofessorcarlson.net/c4dcourse/orientation/orient_unit1/UNICEFC4D... · a communication strategy/initiative to be broader in scope

Table 1: Features of Qualitative & Quantitative Research

The aim is a thorough, detailed description. The aim is to classify features, count them, and construct statistical models in an attempt to explain what is observed.

QUALITATIVE QUANTITATIVE

Good at answering “why” and “how” questions

Ex: Why do mothers resist exclusive breast feeding?; How do people wash their hands in this village?

Good at answering “what” and “how many” questions

Ex: What are the top priorities of decision makers in this region?; How many community members have access to mosquito nets?

Researcher may only know roughly in advance what he/she is looking for.

Researcher knows clearly in advance what he/she is looking for.

Can be useful very during earlier phases of research projects.

Often used during at beginning and end points of projects (baseline and post- measurement).

Number of research participants can be small (less than 20), but also larger if resources allow (hundreds of focus groups).

Number of research participants should be large so that statistical tests can be conducted (+ 500 participants is typical — but can reach into the tens of thousands for large demographic surveys).

The design and protocols can be adapted as the study unfolds.

All aspects of the study are carefully designed before data is collected.

Data is in the form of words, pictures or objects.

Data is in the form of numbers and statistics.

Subjective — individual’s interpretation of events is important, e.g., uses participant observation, in-depth interviews etc.

Perceived as less subjective, seeks precise measurement & analysis of target concepts, e.g., uses surveys, questionnaires etc.

Qualitative data is more ‘rich’, time consuming, and less able to be generalized.

Quantitative data is more efficient, able to test hypotheses, but may miss contextual detail.

Note: This table was adapted from “Qualitative versus Quantitative Research: Key Points in a Classic Debate,” http://www.wilderdom.com/research/QualitativeVersusQuantitativeResearch.html

An additional methods comparison table can be found at: http://web.archive.org/web/20040407031749/http://www.kelcom.igs.net/~nhodgins/quant_qual.html#SOURCES

88 Module 3: Research, Methods, and Practice in C4D

Page 23: Communication for Development (C4D) Learning Courseprofessorcarlson.net/c4dcourse/orientation/orient_unit1/UNICEFC4D... · a communication strategy/initiative to be broader in scope

DISCUSSION

1. Describe the nature, purpose and findings of the research. Refer to any factors, e.g. time, funding or methodology, that you consider may have shaped the scope and/or results.

2. Which of the readings on research from this unit did you find most useful and why?

Discuss a research study you read about, a study you commissioned, designed or participated in, either as a researcher or a respondent. If you do not have experience with research, either as respondent or researcher, or would like to contribute you thoughts on the readings, please answer the second question:

89Module 3: Research, Methods, and Practice in C4D

Page 24: Communication for Development (C4D) Learning Courseprofessorcarlson.net/c4dcourse/orientation/orient_unit1/UNICEFC4D... · a communication strategy/initiative to be broader in scope

[Note: Readings and resources are available on the CD or DVD accompanying this workbook, along with the full citations for all supplied readings.]

RequiRed ReadiNgs: PaRticiPatoRy ReseaRchClacherty, G., and Kistner, J. (2001). Evaluating the Zimiseleni researchers’ project: Participatory research as intervention with “hard-to-reach” boys. Children’s participation: Evaluating effectiveness. International Institute for Environment and Development, Vol. 42. [5 p.]

Chambers, R. (2008). Who counts? Participation and numbers. In Revolutions in Development Inquiry (pp. 105 – 131). [13 p.]

Parks, W., Gray-Felder, D., Hunt, J., & Byrne. A. (2005). Who measures change? An Introduction to Participatory Monitoring and Evaluation of Communication for Social Change. The Communication for Social Change Consortium. [Read Part 2 Participatory Monitoring and Evaluation — What is it? Why use it? — Pages 9 – 13]. [4 p.]

RequiRed ReadiNgs: qualitative techNiquesConducting Focus Group Interviews. Performance Monitoring and Evaluation TIPS. 10. [4 p.]

United States Agency for International Development. Center for Development Information and Evaluation. (1996). Conducting key informants interviews. Performance Monitoring and Evaluation TIPS. 2. [4 p]

United States Agency for International Development. Center for Development Information and Evaluation. (1996). Using direct observation techniques. Performance Monitoring and Evaluation TIPS. 2. [4 p.]

United States Agency for International Development. (2010). Using rapid appraisal methods. Performance Monitoring and Evaluation TIPS. 5. [5 p.]

RequiRed ReadiNgs: quaNtitative techNiquesBabbie, E. (2002). Conceptualization, operationalization, and measurement. In The Practice of Social Research. (10th ed). Belmont, CA: Thomson Learning. [16 p.]

Cresswell, J. (2007). Collecting quantitative data, and analyzing and interpreting quantitative data. In Educational Research: Planning, conducting, and evaluating quantitative and qualitative research (3rd Edition) [Read pages 150 – 176 — and pay special attention to pages 169 – 175, where the concepts of reliability and validity are mentioned]. [25 p.]

READINGS AND RESOURCES

90 Module 3: Research, Methods, and Practice in C4D

Page 25: Communication for Development (C4D) Learning Courseprofessorcarlson.net/c4dcourse/orientation/orient_unit1/UNICEFC4D... · a communication strategy/initiative to be broader in scope

Advocacy, communication and social mobilization for TB control: A Guide to developing knowledge, attitude and practice surveys. [Read pages 4 – 18] World Health Organization, 2008. [15 p.]

RequiRed ReadiNgs: Mixed MethodsStanculexcu, M. and Marin, M. (2011). Using triangulation in targeting social interventions for at-risk-children. Revista de cercetare si interventie sociala, Vol. 33, pp. 131 – 140. [11p.]

Parks, W., Shreshtha, S., & Chitnis, K. (2008). Essentials of Excellence. Researching, monitoring and evaluating strategic communication for behaviour and social change with special reference to the prevention and control of avian influenza/pandemic influenza. [Read the section titled “Toolbox: Basic advice on data collection methods,” pp. 24 – 36]. United Nations Children’s Fund. [12 p.]

oPtioNal ReadiNgsUnited States Agency for International Development. (1996). Center for Development Information and Evaluation. Conducting participatory evaluation. Performance Monitoring and Evaluation TIPS. 1. [4 p.]

Berk, R. A. and Rossi, P.H. (1999). Key Concepts in Evaluation Research. In Thinking About Program Evaluation. Thousand Oaks, CA: Sage Publications [13 p.]

Wynn, B. Dutta, A. and Nelson, M. (2005). A conceptual framework for evaluating health projects in Developing Countries (Chapter 2) and Measurement Issues (Part of Chapter 3). In Challenges in Program Evaluation of Health Interventions in Developing Countries. Santa Monica, CA: Rand Corporation [read pp. 13 – 42 only].

Ulin, P. (2004). Collecting qualitative data: The science and the art. In Qualitative Methods in Public Health (P. Ulin, E. Robinson, & E. Tolley) (71 – 104). Wiley, John & Sons. [27 p.]

Shadish, W. & Steiner, P. (2010) A primer on propensity score analysis. Newborn & Infant Nursing Reviews. 10(1). [7 p.]

Garson, D. (2008). Navigating path analysis: A beginner’s guide. [10 p.]

Valente, T. (2008) Communication network analysis. In A. Hayes, M. Slater, & L. Snyder (Eds.), Advanced Data Analysis Methods for Communication Research (pp. 247 – 273). London: Sage. [26 p.]

Younger, E., Wittet, S., Hooks, C., and Lasher, H., (2001). Immunization and child health materials development guide. [See step 2, “Identify and study your audience,” pp. 11 –18]. Children’s Vaccine Program at Path. [8 p.]

91Module 3: Research, Methods, and Practice in C4D

Page 26: Communication for Development (C4D) Learning Courseprofessorcarlson.net/c4dcourse/orientation/orient_unit1/UNICEFC4D... · a communication strategy/initiative to be broader in scope

Notes

Module 1: Values, Principles, and Concepts in C4D92 Module 3: Research, Methods, and Practice in C4D92

Page 27: Communication for Development (C4D) Learning Courseprofessorcarlson.net/c4dcourse/orientation/orient_unit1/UNICEFC4D... · a communication strategy/initiative to be broader in scope

93Module 3: Research, Methods, and Practice in C4D

Page 28: Communication for Development (C4D) Learning Courseprofessorcarlson.net/c4dcourse/orientation/orient_unit1/UNICEFC4D... · a communication strategy/initiative to be broader in scope

In this unit we will discuss evaluation, a specific type of research designed to measure the effectiveness of our C4D strategies. The research methods from the previous unit are revisited here, to focus on their application for evaluation purposes.

Specifically this unit will provide information on the research process that occurs parallel to the strategy design and implementation process. Three types of evaluation will be covered: formative research, monitoring (sometimes referred to as “process evaluation”), and impact evaluation. In this unit we use the term “monitoring” to describe the collection and analysis of data throughout the project cycle. One can consider “monitoring” a type of constant, on-going evaluation.

Based on the readings and discussion, by the end of this unit, you will be able to:

• Describe the utility of programme evaluation research for C4D strategies.

• Develop arguments for including participatory monitoring and evaluation as part of the C4D process.

• Demonstrate understanding of how to apply impact evaluation techniques to a communication activity/intervention.

The overall research process that occurs parallel to the C4D design and implementation process includes three types of evaluation: formative research, monitoring (process evaluation), and impact evaluation.

Successful evaluation works best when it is a planned process. In other words, if you consider aspects of the strategy to be measured while doing the initial research necessary for developing the communication strategy, you will be more successful in determining whether the strategy has had the intended outcomes.

The linear nature of this course necessitates that information on evaluation is presented after the units on strategy development and implementation. However, in the real world the evaluation process associated with C4D begins very early on, at the same time as the strategy development and continues beyond implementation of activities. Although evaluation is presented here in the final unit, you should be thinking about evaluation from the very beginning of the process.

MODULE 3 / UNIT 4:Programme Evaluation Methodologies

BACKGROUND

INTRODUCTION

LEARNING OBJECTIVES

94 Module 3: Research, Methods, and Practice in C4D

Page 29: Communication for Development (C4D) Learning Courseprofessorcarlson.net/c4dcourse/orientation/orient_unit1/UNICEFC4D... · a communication strategy/initiative to be broader in scope

The table below provides a brief overview of general categories of research, their purpose, and the main questions they answer. Remember: Both qualitative and quantitative research can be designed to be participatory by planning for inclusion of community members and/or stakeholders.

Table 1: Research in the programme cycle — An overview

1. Formative research(Situation analysis, baseline measurement)

TYPE OF RESEARCH GENERAL PURPOSE OF RESEARCH MAIN QUESTIONS ANSWERED

Examines current situation, develops objectives and baselines for subsequent measurement and determines key factors and concepts

Can also be used to develop instruments for follow on research

Where are we now?Is C4D needed? Who needs it, why, how, when and where?

What are the key factors and concepts that need to be explored further?

2. Assessing immediate reactions (pre-testing)

Identifies and assesses essential design features

Will this work?

Do changes need to be made to our communication designs or materials?

How should C4D best be carried out in this case?

3. Monitoring processes and early changes

Monitors inputs and outputs; assesses reach, quality, participant satisfaction, and early indications of behavioural, organizational and social change

How are we doing?

To what extent are planned activities actually realized?

How well is the information provided and dialogue supported?

What early signs of progress can we detect?

4. Evaluating outcomes and impacts

Measures behavioural, organizational and social change results and determines contribution of C4D to these results

How did we do?

What outcomes are observed?

What do the outcomes mean?

What difference did C4Dmake?

Note: This table was adapted from “Research in strategic communication programs,” (Table 1, p. 3) in Essentials for Excellence: Researching, monitoring and evaluation communication for behavior and social change (2008), UNICEF Pacific Office, Fiji.

95Module 3: Research, Methods, and Practice in C4D

Page 30: Communication for Development (C4D) Learning Courseprofessorcarlson.net/c4dcourse/orientation/orient_unit1/UNICEFC4D... · a communication strategy/initiative to be broader in scope

DISCUSSION

Discuss 1. Can community and/or stakeholder

participation strengthen our M & E efforts? If so, how? Or if not, why not?

2. Which forms of research for impact evaluation have you found particularly useful?

3. What experience or suggestions do you have related to including children in the evaluation research process?

4. Which concepts from the readings in this unit do you find most useful for your work?

[Note: Readings and resources are available on the CD or DVD accompanying this workbook, along with the full citations for all supplied readings.]

RequiRed ReadiNgsBertrand, J. (2005). Evaluating health communication programs. The Drum Beat 302. [10 p.] Monitoring/Process Evaluation

Saunders, R., Evans, M. & Joshi, P. (2005). Developing a process-evaluation plan for assessing health promotion program implementation: A how-to guide. Health Promotion Practice 6, 134. [14 p.]

Impact EvaluationHornik, R. (2002). Epilogue. In (R. Hornik) Public Health Communication. Evidence for behavior change. Lawrence Erlbaum Associates, London. [10 p.]

Evaluation and Practical ConcernsLennie, J. & Tacchi J. (December 2010). Evaluating Communication for Development: Trends, Challenges and Approaches. [4 p.]

Rubin, F. (1995). Putting evaluation into practice (Chapter 5). In A basic guide to evaluation for development workers. Some practical considerations. UK: Oxfam. [25 p.]

READINGS AND RESOURCES

96 Module 3: Research, Methods, and Practice in C4D

Page 31: Communication for Development (C4D) Learning Courseprofessorcarlson.net/c4dcourse/orientation/orient_unit1/UNICEFC4D... · a communication strategy/initiative to be broader in scope

oPtioNal ReadiNgsBertrand & Escudero (2002). Compendium of indicators for evaluating reproductive health programs. Chapel Hill, NC: Carolina Population Center. Retrieved from http:// www.cpc.unc.edu/measure/publications/html/ms-02-06.html (18 p.)

Davies, R., & Dart, J. (2004) The Most Significant Change technique: A guide to its use. Self published. [104 p.]

Lennie, J. & Tacchi J. (December 2010). Evaluating Communication for Development: Trends, Challenges and Approaches (Executive Summary). [10 p.]

Linnan, L. & Steckler, A. (2002). Process evaluation for public health interventions and research: An overview. In Process Evaluation for Public Health Intervention and Research. (Chapter 1). New Jersey: Wiley. [24 p.]

Rubin, F. (1995). Some practical considerations when planning an evaluation (Chapter 6). In A basic guide to evaluation for development workers. Some practical considerations. UK: Oxfam. [11 p.]

oPtioNal viewiNgVideo 1: Impact Evaluation 1 Video 2: Impact Evaluation 2 Video 3: Impact Evaluation 3

97Module 3: Research, Methods, and Practice in C4D

Page 32: Communication for Development (C4D) Learning Courseprofessorcarlson.net/c4dcourse/orientation/orient_unit1/UNICEFC4D... · a communication strategy/initiative to be broader in scope

Notes

Module 1: Values, Principles, and Concepts in C4D98 Module 3: Research, Methods, and Practice in C4D98

Page 33: Communication for Development (C4D) Learning Courseprofessorcarlson.net/c4dcourse/orientation/orient_unit1/UNICEFC4D... · a communication strategy/initiative to be broader in scope

99Module 3: Research, Methods, and Practice in C4D

Page 34: Communication for Development (C4D) Learning Courseprofessorcarlson.net/c4dcourse/orientation/orient_unit1/UNICEFC4D... · a communication strategy/initiative to be broader in scope

© Ohio University and UNICEF, 2013. All Rights Reserved.

100

Page 35: Communication for Development (C4D) Learning Courseprofessorcarlson.net/c4dcourse/orientation/orient_unit1/UNICEFC4D... · a communication strategy/initiative to be broader in scope

© Ohio University & UNICEF 2013This material is not licensed for resale or copying without agreement