Measuring empowerment using vignettes

Edoardo Masset

Washington DC, 17 Nov 2014

Motivation

• Specific goal: evaluating the impact of a rural livelihood project on women empowerment

• General goal: is showing how this method can be used in measuring empowerment

• Existing approaches are costly and are biased by subjective perceptions

Empowerment is multidimensional

• Empowerment is a multidimensional concept: Solava and Alkire (2007) found 29 different definitions in the literature

• Indices have tried to capture multidimensionality

• This has resulted in long survey questionnaires

Example: World Bank Empowerment Index

• 8 domains:• Justice• Politics• Public services• Labour• Goods• Private services• Intra-household• Intra-community

• 164 questions

Example: Women’s Empowerment in Agriculture Index

• 6 domains:• Production• Resources• Income• Leadership• Time

• 60 survey questions

Subjective perceptions are not comparable

• People interpret questions in different ways and meanings vary with the context

• Perceptions can be culturally determined and being ‘false’

• Perceptions are not comparable across people, groups or countries

Examples of subjective questions

• World Bank Index: “how much influence do you think you have when the community selects its leaders?”

• Empowerment in Agriculture index: “To what extent do you feel you can make your own personal decisions regarding these aspects of household life if you wanted to”

Andhra Pradesh Rural Livelihood Project

• Goal of reducing poverty by building social capital and empowering women

• Operates through Self-Help Groups to• Channelling funds• microfinance

Women’s empowerment vignette 1. How much freedom/opportunity do you have?

VIGNETTES1. Neelamma takes a loan from the SHG and starts a grocery

shop. Despite his initial opposition, her husband is won round to the idea when he sees the good returns from the business.

2. Manemma takes a loan from the SHG to start a small home business. But her husband argues with her that the money is needed for land improvement, and she reluctantly parts with the loan money.

3. Chandamma wants to take a loan from the SHG of which she is a member to start a tea stall. She could not take the loan because her husband and mother-in-law did not agree.

Men’s vignette1. How much influence do you have in your village?

VIGNETTES1. Davender is in a dispute over some land he claims title to. The

other man claiming this land is a close friend of the Sarpanch, and, lacking connections himself, Davender has little hope of winning the dispute.

2. Vinod is in a dispute over some land he claims title to. He supported the Sarpanch in the last election, bringing him many votes from his family and families of those who work for him. He is optimistic that he will win the dispute.

3. Kiran is in a dispute over some land he claims title to. Both he and the other man in the dispute have some blood relations with leading men in the village. He has some hope that he may win the dispute but realizes it may go either way.

Goal of vignettes

• Find systematic differences in reporting across groups

• Comparing self-reported assessments after purging them of reporting bias

• In the analysis use a two step procedure (HOPIT model)

The Hopit model

• 2-part model (Tandon et al 2003; King et al. 2004)

1.Model reporting behaviour. For example, women of different caste may rate vignettes in different ways

2.Model self-assessments using cut-off points identified in part 1

• Covariates are normally the same in the two parts

Estimation• 2-step: ordered probit of vignettes and

interval regression using estimated cut-off points

• Simultaneous estimation by maximum likelihood. Methods: Programme in R Use GLLAMM (generalised linear latent and mixed

model) in STATA Set the likelihood function in STATA

Examples from the literature• Compare political efficacy in Mexico and China• Compare reported work disability in US and

the Netherlands• Compare job-satisfaction across EU countries• Reported health across demographic groups in

Indonesia, China and India • First study to compare groups with and

without a project

Women’s empowerment and literacy

0.1

.2.3

.4

literate illiterate

Women's power by husband's literacy

no power little powersome power a lot of power

Men’s influence and literacy0

.1.2

.3.4

.5

literate illiterate

Men's power by literacy

no power little powersome power a lot of power

Women’s empowerment and wage work

0.1

.2.3

.4

no-wage wage earner

Women's power by wage earning status

no power little powersome power a lot of power

Men’s empowerment and wage work

0.1

.2.3

.4.5

no-wage wage earner

Men's power by wage earning status

no power little powersome power a lot of power

Women’s empowerment and caste0

.1.2

.3.4

SC ST BC OC

Women's power by caste

no power little powersome power a lot of power

Men’s empowerment and caste0

.2.4

.6

SC ST BC OC

Men's power by caste

no power little powersome power a lot of power

Empowerment and SHG0

.1.2

.3.4

no SHG SHG

Women's power by SHG membership

no power little powersome power a lot of power

  Ordered probit HopitSHG member 0.081* 0.037*  (0.099) (0.712)Scheduled tribe 0.058 -0.173  (0.499) (0.502)Other backward caste 0.076* -0.054  (0.225) (0.675)Other caste 0.041 0.091  (0.622) (0.607)Age 0.020 0.022  (0.123) (0.354)Age square -0.001 -0.001  (0.296) (0.321)Illiterate 0.052 0.202  (0.457) (0.165)Husband’s education 0.083*** 0.012  (0.000) (0.749)Female headed household 0.237** 0.140  (0.003) (0.307)Household size -0.019 -0.052*  (0.134) (0.060)Land size 0.005 0.001  (0.439) (0.993)Agricultural labourer 0.157** 0.099  (0.004) (0.387)Observations 1,431 1,431

Adjusted and unadjusted women self-assessments

• Before adjusting empowerment is correlated with:• SHG membership• Husband literacy• Female headed household• Earning capacity

• After adjusting none of the variables is correlated with empowerment except SHG

• Most differences found are perceptual rather than real

Men’s empowerment

• No expected project impact on men’s empowerment

• Influence correlated with caste, literacy, household size, wage work

• Correlations hold after adjustments of perception bias

• Exceptions are: higher caste understate their influence while OBC overstate their influence

Test of homogeneity in responses

  Female respondents Male respondents

Homogeneity test Full sample Full sample

All covariates 0.002** 0.379

SHG member 0.096* 0.387

Caste 0.568 0.162

Age 0.514 0.494

Illiterate 0.040** 0.100

Husband’s education 0.041**

Female headed household 0.000***

Household size 0.452 0.343

Land size 0.253 0.826

Agricultural labourer 0.835 0.232

Impact of SHG on women’s empowerment

• The data do not contain a valid control group• We follow two approaches:• Instrumented variables• Difference-in-differences

• In both cases, the impact of SHG on empowerment vanishes

Additional question: non-parametric approach

5. How would you compare your freedom/opportunity with that of the women described above? (less than Chandamma, same as Chandamma, more than Chandamma but less than Manemma, same as Manemma, more than Manemma but less than Neelamma, same as Neelamma, more than Neelamma)

SHGs and empowerment

  Coefficient Standard error

observations

SHG instrumented hopit model full sample

0.238 0.844 1,431

SHG instrumented hopit model restricted sample

0.199 0.794 236

Difference-in-difference restricted sample

0.202 0.313 232

Conclusions• Much of the differences in empowerment

observed in our Andhra Pradesh sample are perceptual rather than real

• Empowerment and SHG are correlated but it appears that causality runs from empowerment to SHG rather than the other way

• Vignettes are a simple, unbiased and powerful tool to assess empowerment programmes

Thank you