syed hashemi - qualitative research
DESCRIPTION
Presentation at "Impact Evaluation for Financial Inclusion" (January 2013) CGAP and the UK Department for International Development (DFID) convened over 70 funders, practitioners, and researchers for a workshop on impact evaluation for financial inclusion in January 2013. Co-hosted by DFID in London, the workshop was an opportunity for participants to engage with leading researchers on the latest research methods of impact evaluation and to discuss other areas on the impact evaluation agenda.TRANSCRIPT
An Introduction to Impact
Evaluation: Qualitative Research Syed M Hashemi
1
Ø Qualita've Research -‐ Why? Ø Qualita've Research -‐ How? Ø Prac'cal Considera'ons in Conduc'ng Qualita've Research Ø Review of Findings from Qualita've Research Ø Advantages and Disadvantages of Qualita've Research
Outline of the Presentation
2
Ø Qualita've Research -‐ Why? Ø Qualita've Research -‐ How? Ø Prac'cal Considera'ons in Conduc'ng Qualita've Research Ø Review of Findings from Qualita've Research Ø Advantages and Disadvantages of Qualita've Research
Outline of the Presentation
3
“Not everything that can be counted counts, and not everything that counts can be counted.”
4
Features of Qualitative Research
Ø focuses on the interpretive dimension of the
human experience i.e. meaning, ideas, experience, context and process
Ø explores phenomena hard to capture with numerical data
Ø produces rich information and thick descriptions enhancing understanding and revealing causality
Ø shaped by multiple ethical, economic and political positions
5
Assumptions of Qualitative Approach
Ø The world or reality is not a fixed, single, agreed upon or measurable phenomenon.
Ø There are multiple constructions and interpretations of reality that are in flux and change over time.
6
The Qualitative-Quantitative Divide
Dimension Qualitative Quantitative
1. Type of information
Non-numerical Numerical
2. Type of participant involvement
Active Passive
3. Type of inference methodology
Inductive Deductive
4. Type of data Subjective Objective
7
The Qualitative-Quantitative Divide
Dimension Qualitative Quantitative
5. Type of data collection method
Flexible design Static design
6. Type of analytical framework
Multiple realities Objective reality
7. Primary analytical focus
Processes oriented Cause and effect
8. Role of the researcher
Involved and value laden researcher
Objective and value-free researcher
8
Why Qualitative Research Approach is Necessary?
Ø open-ended questions that reveal new or unanticipated phenomena
Ø holistic interpretation of processes that shape people’s lives
Ø insights into intra-household relations, causes or causal processes
Ø access data on issues that quantitative surveys cannot locate, e.g. domestic violence, illegal migrants, women trafficking
Ø creative and innovative explanatory frameworks
9
Qualitative Research vs. Quantitative Research
• Choice of control group presumes the relevant variables are already known leading to fixed number of variables
Quantitative Research
• Purposive sampling enables close focus on cases and captures diversity of variables
Qualitative Research
Sampling
10
Inductive and Deductive Reasoning
Theory
Hypothesis Observation
Confirmation
Deductive reasoning
Observation
Pattern Tentative
hypothesis Theory
Inductive reasoning
Quantitative Research
Qualitative Research
11
12
Pregnant mother did not get the Tetanus shots
The baby was born without skilled aDendant
Baby turns blue and red with high fever
Mother decides for Health Complex
Grandfather recites Holy Quran and prays
Baby gets worse -‐ taken to a Tradi'onal healer
Baby gets worse-‐ decision to take to UHC-‐ short of money
Slow transport
Baby dies on the way……
Health Services Disease: Tetanus
Power rela'onship
Grandfather stops them
Cultural belief
Health Seeking Prac'ce
Poverty
Infra-‐ structure
13
Ø Capturing the context Ø Understanding characteristics of the target group Ø Understanding process through which change
occurs Ø Understanding why change does not occur
Qualitative Research for Impact Evaluation
14
Ø Qualita've Research -‐ Why? Ø Qualita've Research -‐ How? Ø Prac'cal Considera'ons in Conduc'ng Qualita've Research Ø Review of Findings from Qualita've Research Ø Advantages and Disadvantages of Qualita've Research
Outline of the Presentation
15
Ø sampling Ø rapport building Ø observation Ø data collection Ø data analysis and ethics
Steps in Qualitative Research
16
Ø Intensity sampling-Provides rich information from a few select cases that manifest the phenomenon intensely but are not extreme cases
Ø Stratified purposeful sampling-To illustrate characteristics of particular subgroups of interest; to facilitate comparisons
Ø Snowball or chain sampling (locate one or two key individuals, and then ask them to name other likely informants)-To facilitate the identification of hard-to-find cases
Ø Maximum variation sampling (purposely select a wide range of variation on dimensions of interest)-To document diverse variations; can help to identify common patterns that cut across variations
Steps in Qualitative Research: Sampling
17
Ø familiarizing oneself with the people and area Ø developing a relationship Ø being upfront and clear about your objectives Ø earning trust Ø confidentiality Ø consent
Steps in Qualitative Research: Rapport
18
Ø who is present? -actors
Ø what is happening? -activities -interactions (between different actors, organizations, actors and the public space)
Ø when does the activity occurs? -time, duration
Ø where is it happening? -spatial characteristics
Steps in Qualitative Research: Observation
19
Ø how is the activity organized? -sequences and aim of activities
Ø why is it happening? -listen and ask questions -observe what happens -observe what does not happen
Ø observe your own self -manage your impression -become invisible unless you are actively participating
Observation
20
Data Collection in Qualitative Research
• semi-structured interviews • in-depth interviews • life histories • oral histories and • participant observation
Ethnographic
• focus group discussions • community mapping and
institutional analysis Participatory
method
21
Ø the researcher studies research participants in their natural settings that capture their social interactions and daily activities
Ø involves detailed observation and collection of “unstructured data” by the researcher
Ø narrative constructed only by the researcher Ø active participant involvement Ø reflexive to the learning of the researcher
Key Features of Ethnographic Methods
22
Ø involves participants and the researcher actively analyzing the participants’ natural settings to capture their social interactions and daily activities
Ø involves detailed observation and collection of “unstructured data” both by participants and the researcher
Ø narrative constructed both by participants and the researcher
Ø active and empowering participant involvement Ø reflexive to the learning of the participants and the
researcher
Key Features of Participatory Methods
23
Ethnographic Methods
24
Ø Qualita've Research -‐ Why? Ø Qualita've Research -‐ How? Ø Prac'cal Considera'ons in Conduc'ng Qualita've Research Ø Review of Findings from Qualita've Research Ø Advantages and Disadvantages of Qualita've Research
Outline of the Presentation
25
Participant Observation
Personnel: One highly specialized person or a few Duration: long; few months to a couple of years
Interviews; Life Histories
Personnel: One highly specialized person or a few researchers working directly for the specialized person Duration: medium; few weeks to a few months
FGDs, Community Mapping
Personnel: Trained facilitator(s) Duration: short; 1-3 days for each session, few weeks to write up report I
Practical Considerations on Qualitative Research
26
Ø Qualita've Research -‐ Why? Ø Qualita've Research -‐ How? Ø Prac'cal Considera'ons in Conduc'ng Qualita've Research Ø Review of Findings from Qualita've Research Ø Advantages and Disadvantages of Qualita've Research
Outline of the Presentation
27
The Origin of BRAC Ethnographic studies and other qualitative
research led to BRAC’ mission and strategy and its financial products, the group mechanism and the delivery methodology
Qualitative Research Insights
28
Understanding the Financial Lives of the Poor Ø Small, irregular income Ø Unpredictable cash flows Ø Irregular & seasonal employment opportuni'es Ø Lack of access to markets and infrastructure Ø Low asset ownership Ø High risk and uncertainty
Conventional microfinance inadequate in meeting range of flexible financial needs of the poor
Qualitative Research Insights
29
Microfinance & Women’s Empowerment Ø mobility and visibility Ø control over resources Ø interaction with outside male acquaintances Ø confidence and assertiveness Ø domestic violence Ø changing status within household
Design and delivery mechanism, and not just credit, lead to empowerment
Qualitative Research Insights
30
Qualitative Research: Climate variability in urban slums, DSK-
Shiree Project Objective: Ø to track behavioral changes in clients receiving financial
education training relative to those who have not Methodology: Ø Participant observation of daily activities of clients Ø Participant observation of financial edn plays and training Ø Documentation of attitudes towards financial practices Ø Life histories Ø Focus Group Discussions
Qualitative Research: Financial Education
31
Current Practices • Insufficient understanding of
MFI terms and conditions, death insurance and borrower’s rights
• Infrequent record keeping • Multiple borrowing • Borrowing more than one’s
capacity • Irregular savings practices • Lack of understanding of
returns to investment • Lack of agency in clients in
making financial choices
Desired Practices • Sufficient understanding of MFI
terms and conditions, death insurance and borrower’s rights
• Accurate record keeping • Borrowing from one MFI • Assessing one’s debt capacity
before borrowing • Regular and planned saving
practices • Understanding of returns to
investment • Greater agency in clients in making
financial choices
Financial Education: Behavioral Changes
Poor Financial Decision Informed Financial Decision 32
Qualita3ve Research: Objec3ves and Methods
Objectives: Ø to understand how program inputs address poverty traps
at the household level Ø to capture processes of change as an outcome of program
intervention Methodology:
Ø truncated life histories over the course of 1 year Ø key informant interviews Ø village maps Ø self-evaluation exercise Ø food security questionnaire
Qualitative Research: Graduation Program
33
LESS DYNAMIC | MORE DYNAMIC
Typology of Extreme Poor participants
Typology of Extreme Poor Participants
Constraints
• no vertical social networks
• low earner – dependent ratio
• no cooperative male • lack of previous
experience • health shocks
Resources
• existence of vertical relationships
• high earner-dependent ratio
• cooperative male in household
• previous experience • demonstrating agency
34
Pre-
prog
ram
L
ess D
ynam
ic –
Mor
e D
ynam
ic
Less Dynamic – More Dynamic In program
§ Possessed “success” factors § More resources than
constraints Program “strengthened” their positive trajectory
§ Lacked “success factors” Constraints unaddressed Program failed to bring about change
§ Program succeeded in transforming trajectory Acquired success factors through program and “transformed” their trajectory
§ Negative circumstances - trajectory naturally “devolved”
Program failed to provide enough of a safety net
SLOW CLIMBERS FAST CLIMBERS
Process of Change Matrix
35
Ø new livelihood skills through peer-to-peer experience sharing and weekly refresher training Ø appropriate livelihoods through consultations Ø self-confidence, positive disposition, strong vision for the future through coaching Ø lower incidence of common illnesses due to health messaging
Transformed Trajectories
36
Ø experienced major shocks: migration, death, illness Ø unable to manage assets due to changes in physical capacity Ø low savings due to lengthy bureaucratic procedures Ø absence of personal support and guidance Ø limited access to healthcare during emergencies
Devolved Trajectories
37
Ø Qualita've Research -‐ Why? Ø Qualita've Research -‐ How? Ø Prac'cal Considera'ons in Conduc'ng Qualita've Research Ø Review of Findings from Qualita've Research Ø Advantages and Disadvantages of Qualita've Research
Outline of the Presentation
38
Disadvantages: Ø no statistical representation of population Ø does not establish causality Advantages: Ø understanding process Ø changing narratives based on new insights
mixed methods best for impact studies
Qualitative Research: Advantages & Disadvantages
39
Thank You
40