Social Observatory: Research Themes and Where We Work

Research Themes Where We Work

Adaptation to Climate Change (1)

Food and Nutrition Security (3)

Livelihoods (4)

Poverty and Well-Being (2)

Village Governance (4)

Women’s Empowerment (4)

Bihar

Karnataka

Maharashtra

Odisha

Tamil Nadu

2

Adaptation to Climate Change Can women’s Self-Help Groups increase resilience to natural disasters? Social capital + community networks → Improved disaster management +

sustained recovery?

(why?): Floods, droughts, and cyclones—the intensity and frequency of all of which

are expected to increase as a result of changing climate patterns—pose serious threats to the livelihoods of the rural poor. Cyclone Phailin, for example, which made landfall in India at peak intensity on October 11, 2013, affected up to 12 million people in coastal Orissa. More than half a million people from Odisha and Andhra Pradesh were evacuated to safer places after the cyclone. What are the effects of efforts to aid recovery and rehabilitation of livelihoods following large-scale disasters? Do interventions that focus explicitly on the livelihoods of the poor diminish the immediate impacts of disasters and improve longer-term recovery from these kinds of shocks?

(what?): The Targeted Rural Initiatives for Poverty Termination and Infrastructure (TRIPTI) has been in operation in 10 coastal districts of Odisha since 2009. Cyclone Phailin affected nine of these districts, including three of the worst affected. The project provides credit and supports the development of livelihoods for women who belong to project-facilitated SHGs. In addition to facilitating access to credit and providing livelihood support—key factors in aiding recovery—SHGs may foster social capital that allows them to mobilize people and resources to mitigate risk and accelerate recovery. The Social Observatory designed an impact evaluation that examines how livelihoods-focused interventions help mitigate the impact of and hasten recovery from disasters.

(how?): Assessing how program interventions aid recovery and rehabilitation after

natural disasters is challenging, because doing so requires data on both project and comparable nonproject areas before a disaster. Given that the site of a disaster is not known before it strikes, the likelihood of having such data is extremely low. Data collected in 2011 as part of the TRIPTI baseline survey covered both project and nonproject blocks. (The survey was conducted as part of an ongoing Social Observatory–supported impact evaluation of TRIPTI, which used a regression discontinuity design.) Because Phailin affected both project and nonproject blocks covered by this evaluation, this baseline survey provides a unique opportunity to examine whether and how community-based livelihoods approaches help mitigate the impact of and hasten recovery from disasters. Using the variation in the intensity of exposure to Phailin, as well as survey data on affected TRIPTI districts from before and after Phailin, this evaluation will (a) estimate the impact of belonging to a SHG on livelihoods, income and expenditure, and food security and health after Phailin and (b) examine whether such participation can aid recovery only up to a

3

threshold of severity of cyclone impact. The Social Observatory’s embedded approach to research allowed it to identify this unique opportunity and to implement a disaster recovery–focused follow-up survey to assess impacts.

(project status): Baseline: Completed 2011 Endline: Survey completed 2014; data under analysis Project completion: June 2015

(outputs): Forthcoming

(principal investigators): Paul Christian, Research Associate, Charles Dyson School of Applied Economics and Management, Cornell University Eeshani Kandpal, Economist, Development Economics Vice Presidency (DEC), World Bank Nethra Palaniswamy, Coordinator, Social Observatory Vijayendra Rao, Head, Social Observatory

4

Food and Nutrition Security Do the poor pay higher prices for food? Women’s Self-Help Groups → Access to bulk purchases of food + lower prices

for the poor?

(why): Poor households may pay higher unit prices for identical goods because they

are unable to leverage discounts that come from purchasing large quantities of food at one time. Previous surveys, including the National Sample Survey and the Bihar Rural Livelihoods (JEEViKA) Project, were unable to quantify the importance of bulk discounting because they could not distinguish between price variation associated with quantity and quality. Can buying food in bulk benefit poor people? Do women’s SHGs facilitate such purchases?

(what?): An extensive and carefully contextualized food consumption questionnaire

was developed to finely track categories of food consumed in rural Bihar, India. This survey was supplemented by surveys of grocery stores and “ration” shops affiliated with the public distribution system. Data from these surveys were used to compare samples of households from randomly assigned villages that participated in the JEEViKA Project intervened and a control group of villages that did not. The results should shed light on whether women’s SHGs make it more likely that poorer households benefit from lower food prices by buying food in bulk.

(how?): A special purpose survey was designed that seeks to match household

reports of food security with audits of markets to identify the following:

the size of potential bulk discounts

the effect of discounts on nutritional status

the characteristics of households, including poverty levels, that access bulk discounts

the effect of JEEViKA on food prices in villages. (project status):

Data collection: 2014 Food markets dietary recall survey: November 2014 Project completion: February/March 2015

(outputs): Forthcoming

(principal investigators):

Paul Christian, Research Associate, Charles Dyson School of Applied Economics and Management, Cornell University Vijayendra Rao, Head, Social Observatory

5

Do community-based approaches reduce anemia? Community-based approaches + local public health facilities → Reduced anemia?

(why?): Anemia is a critical measure of women’s nutrition and has important intergenerational effects. In Tamil Nadu, in southern India, more than half of all pregnant women and women of reproductive age (15–49) are anemic. Despite these high rates of anemia, only 43 percent of pregnant women consumed the recommended iron and folic acid tablets for at least 90 days during their pregnancy, according to the 2005–06 National Family Health Survey, and the extensive network of health facilities has not been able to adequately address this problem, according to a 2005 report by the Planning Commission. Can community-based programs that pay attention to local contexts and try to affect healthcare behavior through information campaigns address this problem?

(what?): The Tamil Nadu Empowerment and Poverty Reduction (Pudhu

Vaazhvu) Project is leveraging its network of SHGs and other community-based organizations to implement a community-based, information-centered approach to delivering preventive and referral health care–related services for women. The project needs evidence on the effectiveness of the approach in reducing anemia rates. This evaluation will compare the project’s community-based modality of targeting health and nutrition outcomes with top-down programs that target women’s health and nutrition in nonproject areas. It will indicate whether the project’s health care intervention has the potential to increase demand for health-seeking behavior from women and reduce anemia.

(how?): This evaluation will piggyback on the impact evaluation of the project’s

core intervention, using the same identification strategy and evaluation samples. The addition of data on anemia (from hemoglobin tests) will allow the Social Observatory to identify the impact of this intervention.

(project status):

Data collection: Expected 2015

(outputs): Forthcoming

(principal investigators): Shareen Joshi, Assistant Professor, Edmund Walsh School of Foreign Service, Georgetown University Nethra Palaniswamy, Coordinator, Social Observatory, World Bank

6

Do community-based food security programs work? Start-up loan + bulk procurement of staple foods → Reduced borrowing + better nutrition

(why?): Bihar, in northern India, has some of the highest rates of food insecurity

and malnutrition in the world. Can community-based programs reduce them?

(what?) To help members of Self-Help Groups access affordable staple foods, the Bihar Rural Livelihoods (JEEViKA) Project introduced the Food Security Fund. The project provides grants of Rs 100,000 (about $2,000) to village officers to start the fund. Members of the village organization are surveyed to assess their food needs and estimate the quantities they want to purchase. Based on this demand, the village organization chooses one variety of rice or wheat and solicits bids from local retailers. Village organization members accept the grain as a loan to be repaid after three months at no interest. After the loans are repaid and the fund is replenished, this three-month cycle of estimation, purchase, and sale begins anew. The goal of the program is to help members accurately estimate and procure their food without incurring high-interest loans and access purchasing discounts through bulk purchases of grain.

This evaluation answered three questions: (a) does this program alleviate food insecurity among eligible members, as measured by self-reports of food insecurity and anthropometric measurements of children; (b) which members benefit most from the program; and (c) how does the choice of grain to buy determine whether the program is most appealing to the poorest or least poor members of the village organization?

(how?): The Social Observatory team designed a cluster randomized control

trial. Groups of villages from the same geographical areas in six blocks of Bihar were randomly assigned to participate in the program, while other groups of villages were randomly assigned to wait to start participating until the evaluation was complete. Food grains in Bihar vary considerably by quality and price. Whether the

village provides a high-cost, high-quality grain or a low-cost, low-quality grain may determine whether the program appeals most to relatively poor or relatively wealthy members.

(project status): Baseline: August 2012 (before establishment of the Food Security Fund) Midline: January 2013 Endline: October 2013

(materials): Food Security Study

(findings): This evaluation yielded rigorous evidence of the effectiveness of the program. This evidence is currently being shared with project staff to better understand the program’s successes and suggest how to better meet the needs of the poorest beneficiaries.

7

(outputs): PAPER: The Distributional Consequences of Group Procurement

develops an economic model of the interaction between collective choice and program targeting. It explores the model empirically through the data collected for this project. A preliminary version of this paper was presented at the Northeastern Universities Development Consortium conference at Harvard in 2013. It is being developed as a World Bank Working Paper.

(principal investigator):

Paul Christian, Charles Dyson School of Applied Economics and Management, Cornell University

8

Livelihoods

Participatory projects are driven by the hope that involving beneficiaries in the design and management of local-level interventions will lead to better and more equitable development. In recent years, they have been a popular way of delivering foreign aid. Over the past decade, the World Bank invested $85 billion in participatory community driven-development projects. These interventions—which have a variety of goals, from improving the delivery of local infrastructure and public services to providing microcredit and empowering women—represent a multidimensional approach to improving welfare and reducing poverty. Community-driven development projects also increasingly include livelihoods components that invest in skill development and links to markets and widen the scope of anti-poverty interventions.

Despite the popularity of these projects, the empirical evidence on whether they work is weak. The need for evidence is particularly pressing in India, where the World Bank has invested $1.2 billion in such projects and the government’s National Rural Livelihoods Mission expects to invest an additional $5 billion over the next decade. Evaluating livelihoods-focused community driven-development projects is particularly challenging, for a variety of reasons. First, the core participatory intervention of creating networks of SHGs often has a long implementation history, with the involvement of different donors and governments. Second, once SHGs have been created, projects begin to implement a variety of “vertical” interventions that use the networks to deliver a variety of services. Third, a significant proportion of the geographic area under these projects has ongoing operations that are almost complete, making it difficult to find good controls. To deal with these challenges, the Social Observatory evaluated the core intervention by examining the impact of creating networks of women’s SHGs as well as various vertical subinterventions in three states: Bihar, Odisha, and Tamil Nadu. It combined the most rigorous evaluation design feasible with “quick and dirty” evaluations to assess the impact of this important portfolio of projects.

The Social Observatory designed and implemented five evaluations of the core SHG intervention. These evaluations use matching methods for retrospective evaluations of the first phase. To evaluate the Bihar Rural Livelihoods (JEEViKA) Project and the Tamil Nadu Empowerment and Poverty Reduction (Pudhu Vaazhvu) Project, the Social Observatory used regression discontinuity designs, a method also used to evaluate the second phase of the Tamil Nadu project, and The Orissa Rural Livelihoods Project (TRIPTI). The second phase of the JEEViKA project in Bihar was evaluated with a carefully designed randomized control trial. To

9

shed light on the process behind the impact (or lack of impact), the quantitative evaluations in Bihar and Tamil Nadu were integrated with qualitative data and analysis. In the JEEViKA evaluation in Bihar, behavioral experiments are being used to assess the impact on the difficult to measure issue of women’s voice and agency.

Do Self-Help Groups reduce poverty and empower women?

Evidence from the JEEViKA project in Bihar and the Pudhu Vaazhvu project in Tamil Nadu

Forming women’s Self-Help Groups → Reduced poverty and greater

empowerment

(why?): Several livelihoods-focused community-drive development projects in India

completed their first phase of operations in 2011. None of them had a credible impact evaluation. Despite this paucity of evidence, the project interventions were scaled up both within states and across the country through the National Rural Livelihoods Mission. This scale-up was based largely on anecdotal evidence and poorly designed program assessments.

Two of the projects that were slated for significant scale-up were the Bihar Rural Livelihoods (JEEViKA) Project and the Tamil Nadu Empowerment and Poverty Reduction (Pudhu Vaazhvu) Project. In Bihar the project envisaged scaling up to all blocks of the 34 districts in the state, covering 12.5 million households. In Tamil Nadu, the project planned to scale up to cover almost 4 million households. As more rigorous (and forward-looking) evaluations could not be implemented in time to inform the expansion in these projects, there was a strong evaluation need for “quick and dirty” evidence on this project’s portfolio. The Social Observatory team used quantitative evaluations, which use matching methods to generate credible and first-time evidence on the impacts supported by this livelihoods approach. These evaluations are one component of a comprehensive learning system in both projects. They are part of a set of impact evaluations that include a more rigorous (randomized control trial– or regression discontinuity design–based) evaluation of their second phase, as well as separate impact evaluations that examine several subinterventions within these projects. This set of evaluations, along with insights from qualitative fieldwork and a series of behavioral experiments, will inform learning in both projects.

(what?): JEEViKA and Pudhu Vaazhvu are typical multi-intervention community-drive development projects. Both address the multiple dimensions of household poverty by empowering project-facilitated SHGs. The core economic mandate of these projects is to facilitate access to credit and better livelihoods for women who belong to these groups. Village organizations, which are most often federated from these groups, then implement a set of interventions that range from attempting to

10

improve food and nutrition security to improving political participation and local civic action and increasing women’s voice and agency.

(how?): The retrospective evaluations measure the impact of this interrelated set of interventions on women’s empowerment, political participation and local civic action, and household poverty and indebtedness that these projects seek to affect. The JEEViKA evaluation covered 4,000 households from 200 project and 200 nonproject villages. The PVP evaluation covered 3,692 households (half treatment and half control) from 268 villages drawn from 20 blocks in 10 program districts. The retrospective evaluation used propensity-scoring methods to identify a robust counterfactual for these evaluations. Phase 1 project areas in both projects were selected based on demographics (the share of the population living below the poverty line, belonging to scheduled castes and tribes and/or meeting other criteria for “backwardness.”1 Census data were used to measure the components of this criteria. The sampling strategy for the evaluations was to replicate this project area selection as closely as possible using preprogram census data and to match them to very similar nonproject areas using the same criteria. In Tamil Nadu a qualitative component was integrated with the quantitative evaluation in order to understand processes of impact.

(project status of JEEViKA): Survey: Ongoing

(findings): JEEViKA had a positive impact on both credit and women’s empowerment. The burden of debt was reduced, and project beneficiaries shifted toward borrowing for productive purposes. Beneficiary women also displayed greater mobility, decision making, and confidence to engage in collective action. Although there was no detectable change in livelihood patterns, beneficiary households reported increased asset holdings, better sanitation practices, improved food security conditions, and regularity in savings.

(outputs): Forthcoming (principal investigator of JEEViKA):

Upamanyu Datta, Economist, Social Observatory, World Bank

(Project status of PVP): Survey: December 2012–March 2013

1 The Indian government uses the term backwardness to define a specific group based on economic and social indicators.

11

(findings): The Pudhu Vaazhvu Project had significant positive impacts across

the multiple dimensions it sought to affect. Women’s empowerment and agency

increased in both local public action and intra-household decision-making. In

project areas, the debt burden facing households fell and measures of economic

welfare at the household-level improved. In line with its focus on targeting the

excluded poor, households from scheduled castes in program areas had more

diverse livelihoods portfolios than scheduled castes in nonprogram areas, participated

more in local deliberative forums, and reported greater changes on some measures of

women’s empowerment.

(output): Paper: (principal investigators of Pudhu Vaazhvu):

Madhulika Khanna, PhD student, Department of Economics, Georgetown University Nishtha Kocchar, PhD student, Department of Economics, Georgetown University Nethra Palaniswamy, Coordinator, Social Observatory, World Bank

How do Self-Help Groups empower women and reduce poverty—and by how much do they do so? Evaluations of Bihar, Odisha, and Tamil Nadu

Evidence from Bihar

Forming women’s Self-Help Group → Reduced poverty + reduced debt + greater empowerment?

(why?): The Bihar Rural Livelihoods (JEEViKA) Project is a World Bank–supported community-driven antipoverty project implemented by the government of Bihar, one of India’s poorest states. It attempts to reduce poverty and enhance gender equality by forming a network of women’s SHGs, which are then federated into village organizations and cluster-level federations. The core idea is to use the network of women’s groups as a “highway” on which to roll out a variety of “vertical” antipoverty interventions, such as microfinance, food security, and nutrition programs. By 2022 JEEViKA plans to be in operation in all blocks of Bihar’s 34 districts, covering 12.5 million households. Given the scope of the project, and the paucity of evidence on the effectiveness of women’s SHGs as a mechanism for reducing poverty, an impact evaluation of the project was required.

(what?): This intervention seeks to understand the mechanisms underlying the program’s observed impact (or lack thereof).

12

(how?): This intervention combines a randomized control trial using survey data of a phased roll-out of the intervention with in-depth qualitative ethnographic analysis of a subset of the qualitative sample to understand the mechanisms underlying the observed impact (or lack thereof). Given that a key goal of the project is to reduce gender inequality in Bihar—one of the world’s most gender-unequal regions—the evaluation uses a variety of behavioral experiments to measure women’s voice and agency to test whether the intervention is able to improve them.

(project status): Baseline: July–September 2011 Endline: September 2014 Project completion: September 2015 ((outputs)): Forthcoming

(principal investigators):

Upamanyu Datta, Economist, Social Observatory, World Bank Vivian Hoffmann, Research Fellow, International Food Policy Research Institute Vijayendra Rao, Head, Social Observatory, World Bank

Self-Help Groups → Greater voice and agency + shift in social norms + bridging

of caste inequalities + reduction in patriarchy

(what?): Does JEEViKA reduce social inequality, increase the responsiveness of important public institutions, and foster greater gender equity? This four-year in-depth qualitative study aims to understand the processes and mechanisms of change and the mediating influences that facilitate or obstruct change in women’s empowerment and agency.

(how?): In each of 10 sample villages, researchers collected extensive data on baseline conditions to gain a comprehensive understanding of the layout of the village (through transect walks, key informant interviews, household mapping, and several participatory rural appraisal methods). Every three to four months during the study period (2011–15), a team of four researchers from Praxis, a Patna-based think tank, collected data in each village. The team collected data through conversations and in-depth unstructured interviews with program participants and nonparticipants; passive observation of group meetings and interactions at several levels (village, block, district); focus group discussions with participants; and structured interviews with JEEViKA staff at all levels in all villages.

(project status):

First cycle started: September 2011 Final cycle completed: March 2015

(outputs): Forthcoming

13

(principal investigators):

Shruti Majumdar, Sociologist, Social Observatory, World Bank Vijayendra Rao, Head, Social Observatory, World Bank Paromita Sanyal, Assistant Professor, Department of Sociology, Cornell University

Forming Women’s Self-Help Groups → Reduced discrimination against women?

(what?): Behavioral experiments are being used to measure whether women are less likely to be heard in private and public settings than men (“epistemic discrimination”). The study tests whether JEEViKA is able to reduce epistemic discrimination and thus increase women’s voice and agency both within the household and in the public sphere.

(how?): The study included two experiments and a survey questionnaire. The first experiment was a software-based test developed to capture implicit gender-biased attitudes. The second was a vignette presented to respondents by showing them a video followed by some related questions to solicit their views, perceptions, and attitudes. Respondents were also asked some questions about their lives, aspirations, participation in decision making inside and outside their households, family health conditions, and trust and confidence toward society and neighborhood. The experiments were conducted in 70 villages in which JEEViKA had been working for two to three years and 70 control villages matched with propensity scores derived from exogenous village-level characteristics.

(project status):

Data collection: 2012–13

(outputs): Epistemic Discrimination: Measuring Women’s Agency in Bihar (forthcoming); Does JEEViKA Change Women’s Agency (forthcoming)

Principal investigators:

Karla Hoff, Senior Research Economist, Development Economics Research Group, and Director, World Development Report 2015 Tauhidur Rahman, Associate Professor, Department of Agriculture and Resource Economics, University of Arizona Vijayendra Rao, Head, Social Observatory, World Bank

Evidence from Odisha

Women’s Self-Help Groups → Reduced poverty + reduced debt + greater

empowerment?

14

(why?): The is a participatory project that provides credit to and supports the

livelihoods of groups of women who belong to project-facilitated SHGs. The project’s objective is to increase the productive potential, and therefore the socioeconomic status, of the poor. To do so, it invests in processes that empower the poor by building the institutional capacity of community-based organizations. The creation of community-level institutions, which are federated from project-created SHGs at the level of the village, is the core of this process of institution building. The project then facilitates access to credit by providing SHGs with grants. Does building institutions of the poor enable poor people to negotiate with service providers (government, private sector, and civil society) for better service delivery?

(what?): The evaluation—one of three rigorous evaluations that examine the same multidimensional project design in different states and therefore implementation contexts—assessed the effect of the project on household welfare (as measured by household indebtedness, livelihoods portfolios, and consumption expenditures) and local collective action. Evaluating the same design in different implementation contexts should confirm external validity to some extent.

(how?): This evaluation used a regression discontinuity design. The roll-out of TRIPTI program blocks was determined by a scoring criterion. The score was based on block-level development indexes, the total population, and the population of people from scheduled castes and tribes. The weighted block-level score determined the assignment of blocks to the program (treatment). Using this score, the four most “backward” blocks within a district were assigned to treatment. All villages within a block received treatment. The regression discontinuity design exploited this program assignment rule to create a counterfactual. The evaluation covered 400 villages from 20 blocks in the 10 TRIPTI districts.

(project status): Baseline: Completed 2011 Endline: Completed 2014 Status: Data being analyzed

(outputs): Forthcoming

(principal investigators): Shareen Joshi, Assistant Professor, Edmund Walsh School of Foreign Service, Georgetown University Nethra Palaniswamy, Coordinator, Social Observatory, World Bank

15

Evidence from Tamil Nadu

Women’s Self-Help Groups → Reduced poverty + reduced debt + greater

empowerment?

(what?): Pudhu Vaazhvu Project is an antipoverty project that implements

multiple project interventions. Implemented through a community-based village organization (VPRC), it is designed to target the multiple dimensions of poverty. The VPRC facilitates access to affordable credit, helps build market linkages in an attempt to improve livelihoods, and implements a skills program that trains rural youth and matches them to suitable job opportunities. It also places significant emphasis on making the local government work for the rural poor. In this role of public action, it facilitates access of the poor to safety net and social service programs and attempts to improve accountability of local governments to the poor. This evaluation covers the second phase of the project, launched in 2012. It examines the impact of this interrelated set of interventions on the multiple measures of women’s empowerment, political participation and local civic action, and household poverty and indebtedness that these projects seek to affect. The evaluation was designed as part of a set of evaluations that seek to generate evidence on the World Bank’s portfolio of livelihoods projects in India. At the time of the design of the retrospective evaluation of the first phase of this project, its second phase had not yet been initiated, providing the opportunity for a more rigorous evaluation of the impacts of this program.

(how?) This evaluation uses a regression discontinuity design. The roll-out of the program into blocks, the key unit of intervention, was determined using a scoring criterion. This score—determined based on the infant mortality rate, the rural female literacy rate, the percentage of the population from schedule castes and tribes, a score for industrial “backwardness,” and the percentage of unirrigated area to total cultivable land—was used to identify the four to six poorest blocks, which were assigned to the program. The design exploits this program assignment rule to create a counterfactual. The evaluation covers 20 blocks from 10 project districts.

(project status): Baseline: Completed 2012 Endline: Expected December 2015

(outputs): Forthcoming

(principal investigators): Shareen Joshi, Assistant Professor, Edmund Walsh School of Foreign Service, Georgetown University Nethra Palaniswamy, Coordinator, Social Observatory

16

Poverty and Well-Being What role do quantity discounts play in providing equitable access to food? Poverty + purchasing small quantities of food → higher food prices +

malnutrition?

How can well-being be measured? Tracking multidimensional poverty → Tools for better policy?

(why?): Self-Help Groups composed of women have long been implementing

credit and livelihoods interventions in Maharashtra and Odisha. The National Rural Livelihoods Program (NRLP) reorganized these groups to implement other similar interventions. This study seeks to measure and track project outcomes under the NRLP, a task that was complicated by the extensive presence of preexisting SHGs.

(what?): The National Rural Livelihoods Program (NRLP) reorganized women’s

SHGs to implement other similar interventions in Maharashtra, India’s second-largest state, and Odisha. To track well-being and living standards, it is using the Well-Being and Living Standards (WELLS) panel. Emphasis will be placed on measuring the welfare and empowerment of women and extremely poor and vulnerable groups. Indicators used will include income, consumption expenditure, asset base, savings, credit access and terms of access, debt portfolio, food security, health and nutrition status, and access to employment opportunities. The WELLS will also measure village-level social and economic inequality, empowerment, local public action, and changes in the quality of public goods and services. The WELLS will also measure networks. The dynamics of group formation and the role of network ties as collateral is not well understood. In addition, mapping networks and their evolution as the NRLP expands over time in Odisha will help identify the role of social capital in participation in SHGs and the relationship between networks, SHGs, and female empowerment.

(how?): WELLS panel surveys will be conducted in a representative sample of the project area and in matched nonproject areas to track these outcomes. As the NRLP takes a multidimensional approach to poverty reduction, these surveys will track both monetary and nonmonetary measures of household welfare. In order to map networks, the WELLS household surveys in Odisha will include a network listing component in which each respondent will be asked to list all friends and relatives in the village. The listing will also record the nature (friend or relative) and strength of ties (frequency of interactions). This network listing will be collected at baseline and endline, in the hope of shedding light on the role of social networks in SHG participation and the role of participation in a SHGs in the

17

formation of new ties.

(project status, Maharashtra): Tracking survey: Ongoing from 2014 Project completion: Open ended ((outputs)): Forthcoming

(principal investigators, Maharashtra):

Ashwini Deshpande, Professor, Delhi School of Economics Anirudh Krishna, Professor, Department of Public Policy, Duke University

(project status, Odisha):

Tracking survey: Expected start date 2015

(outputs): Forthcoming

(principal investigator, Odisha): Eeshani Kandpal, Economist, World Bank

18

Village Governance Can external policy interventions strengthen civic participation? Citizen participation + active facilitation → More accountable local governments?

(why?): Billions of dollars of funding are allocated to programs that induce citizen

participation to improve the quality of government at the local level: the World Bank alone spent about $85 billion on such programs over the past two decades. How well do these programs actually work? Can civic participation be externally induced? Can policy interventions deepen democracy? These questions lie at the heart of development “as freedom,” as Nobel laureate Amartya Sen has noted.

(what?): The Anatomy of Failure: An Ethnography of a Randomized Trial to Deepen Democracy in Rural India is an ethnography of a randomized control trial to deepen democracy in rural India. It examines the impact of a two-year effort to deepen democracy in the poor and arid region of north Karnataka, in south-central India.

(how?): The intervention was evaluated using a mixed-method design. Survey data were collected over two rounds on a sample of 200 villages, with half randomly assigned to be treated by the intervention. The study makes a methodological contribution to the literature on impact evaluations by combining a randomized control trial with in-depth ethnographic research to understand the mechanisms behind the changes in outcomes measured by the two rounds of quantitative surveys. Ethnographic methods were used to track a 10 percent subset of the quantitative sample over a four-year period.

(findings): The study demonstrates that "thick description" to uncover the process of change using careful and detailed qualitative work can add value to standard impact evaluations. The quantitative data show no impact from the intervention. In contrast, household and village survey data show considerable improvement across a wide variety of governance and participation indicators over time, although the differences in the changes between treatment and control villages are not statistically significant. The detailed qualitative analysis helped unpack the reasons why the intervention failed to have the desired impact. It highlights the role of variations in the quality of facilitation, the lack of top-down support, and difficulties confronting the stubborn challenge of persistent inequality. The qualitative investigation also uncovered subtle treatment effects that are difficult to observe in structured surveys.

(current status):

Baseline: Completed 2007 Final cycle: Completed 2012

(output): The Anatomy of Failure: An Ethnography of a Randomized Trial to Deepen Democracy in Rural India, Vol. 1 (World Bank Policy Research Working

Paper No. 6958)

19

(principal investigators):

Kripa Ananthpur, Professor, Madras Institute of Development Studies

Kabir Malik, Economist, World Bank

Vijayendra Rao, Head, Social Observatory, World Bank

Do Women’s Self-Help Groups help equalize the participation and voice of men and women in village meetings? Women’s Self-Help Groups + deliberative forums → Increased women’s

participation in local public action?

(why?) The gender gap in political participation has been a longstanding policy

concern. In response, policy makers have sought to mandate women’s inclusion in elected bodies through quotas or reserved seats. The effect of such mandates has been mixed at best, for several reasons. For one, local governments could be subject to capture by powerful local elites, who reinforce existing gender inequities. Studies that find positive effects of women’s quotas attribute these changes to the incumbents’ preferences rather than to women’s participation itself. Rates of female political participation remain low, but not, apparently, out of lack of desire for change or policy preferences that favor men. Ethnographic accounts from India suggest that under certain conditions, women are willing and able to advocate for victims of domestic abuse, to institute anti-alcohol ordinances, and to demand preferred public goods in their local village assemblies. In the past few decades, increasing attention has been paid to the social, economic, and political welfare of women around the world. Although the impact of livelihood projects and educational campaigns on women’s economic welfare have been the focus of development practitioners, little work has been done to understand how these efforts affect the political empowerment of women.

(what?) This evaluation seeks to estimate the effect of a livelihoods program that works with local governments on women’s participation in local politics and civic action. The program, implemented in the state of Tamil Nadu, focuses primarily on women’s economic empowerment. The study examines whether it can also facilitate their participation in local politics and civic action. The evaluation also examines whether such participation reflects women’s underlying preferences and whether this political engagement fosters public action for local development.

(how?) : This evaluation was designed after the project had already been in

operation. It used propensity score matching methods in a cross-sectional sample of 100 panchayats to estimate the impact of the program on gender-specific outcomes. The key data source is transcripts from village assembly

20

(gram sabha) meetings. Text analysis methods are used to evaluate the frequency and content of women’s participation, as well as responses to their concerns. Survey data will be collected on women’s preferences and participation in local village assemblies. Three hundred enumerators collected the data on a single day—January 26, 2014, India’s Republic Day, when all village governments in Tamil Nadu must hold a village meeting. The key data collection challenge was to identify who participated not just by gender but also by other markers of disadvantage, such as caste. Given these needs and the time constraints, Pudhu Vaazhu community professionals who are frontline field agents—were trained as enumerators.

(current status): Data collection: Completed January 26, 2014 Status: Data being analyzed

(outputs): Forthcoming

(principal investigators):

Nethra Palaniswamy, Coordinator, Social Observatory, World Bank Ramya Parthasarathy, PhD student, Department of Political Science, Stanford University Vijayendra Rao, Head, Social Observatory, World Bank

Which does more to improve participation in local government—fostering the creation of Self-Help Groups or setting aside seats for women in legislative bodies? Local economic empowerment initiatives + strong mechanisms of local

accountability → Improved women’s political participation?

(why?): The gender gap between men and women is starkest in the political

arena, where women are considerably less likely than men to run for or be elected to office. The two main explanations for this gap focus on cultural norms relegating women to the household and lack of economic opportunity and education, which curtail women’s bargaining power in the home. To address the problem, many developing countries have established mandated representation for women, with the hope that bringing women into office will only increase the legitimacy of governing bodies and that such bodies will be more responsive to women’s needs. At the same time, development policies have increasingly targeted women for livelihoods training, savings and credit associations, and other such programs. Although both forms of intervention—“top-down” interventions to bring women into office and “bottom-up” interventions to empower them to be effective participants in the political process—have

21

received considerable scholarly attention, evidence for their efficacy has been mixed. Moreover, no work has looked at the relative impact of and potential interaction between these types of interventions in the same context.

(what?): This study evaluates the effect of setting aside political offices for women in the Tamil Nadu Empowerment and Poverty Reduction (Pudhu Vaazhvu) Project, a participatory livelihoods project aimed at both reducing poverty and empowering women. It examines the extent to which economic constraints limit the ability and willingness of women to participate. Exogenous variation in the gender of the panchayat president (as a result of the mandated cycle of reservations) reveals whether (a) the presence of a woman incumbent increases women’s overall participation and (b) female incumbents are more responsive (in terms of providing public goods and targeting female-headed households, for instance) to women’s increased participation.

(how?): Two identification strategies are used to estimate the causal effect of the political and economic interventions on women’s political empowerment and governmental responsiveness. To estimate the effect of the women’s reservations, the study relies on the exogeneity of the assignment process. To identify the effect of the participatory livelihoods intervention, the study relies on a regression discontinuity design. Selection into the program was not randomized; it was based on a scoring criterion. Because selection into treatment is as-if-random just around the cutoff eligibility score, the study can compare villages just above and just below the cutoff to recover the casual impact of the intervention on the outcomes of interest. The evaluation will cover 350 panchayats and 4,500 households drawn from 10 districts in Tamil Nadu.

(project status):

Baseline data collection: Completed 2012 Endline: December 2015

(outputs): Forthcoming

(Principal investigator): Ramya Parthasarathy, PhD student, Department of Political Science, Stanford University

Can incentives improve the performance of democratically elected village governments? Community-based organizations + local government incentives → Propoor

governance

(why?): Local governments are often captured by local elites. To address this

problem, several participatory project interventions set up community groups that include the poor. This approach is based on the hope that community-based organizations of the poor are likely to ensure that program benefits target the

22

poor. But powerful local interests could capture these groups, and working in parallel with local government limits the ability of these programs to deliver broad-based improvements in local accountability and service delivery. This dichotomy—between encouraging participation of the poor and then limiting its more broad-based use—is likely to limit the scope of what large-scale participatory projects can deliver. The Pudhu Vaazhvu Project in Tamil Nadu implements a participatory project that explicitly addresses this dichotomy. It first forms community-based groups to implement the program (to limit capture). It then requires that these groups work in partnership with rather than parallel to local governments, through a system of performance-based financial incentives for local government. The goal is to make local governments work with these community groups and to target local services and programs to the poor. These performance-based incentives are an increasingly popular way to deliver aid and implement development programs. Whether they actually engender pro-poor governance is largely unknown.

(what?): This evaluation examines whether local governments that work with participatory projects can be incentivized to target the poor. It focuses on how different types of incentives, which draw on different motivations, influence pro-poor targeting. In particular, it compares monetary incentives (which draw on more instrumental motivations) with nonmonetary incentives (which build on intrinsic motivation) to better understand how governance can be made pro-poor.

(how?): Using a randomized control trial, this evaluation examines whether local

governments can be incentivized to work for the poor and identifies which types of incentives work. Groups of villages from the same geographical areas in 10 blocks of Tamil Nadu were randomly assigned to receive or not receive financial incentives. The baseline survey covered 3,600 households from 200 panchayats. Project monitoring data will be used along with these survey data.

(project status):

Baseline: Completed 2012 Endline: Expected December 2015

(outputs): Forthcoming

(principal investigators): Ghazala Mansuri, Lead Economist, World Bank Nethra Palaniswamy, Coordinator, Social Observatory, World Bank Vijayendra Rao, Head, Social Observatory, World Bank Slesh Shrestha, Assistant Professor, Department of Economics, National University of Singapore

23

Women’s Empowerment

Can community-driven development change the social meaning of money?

Increasing access to credit by poor women → More egalitarian discourse around

money?

(why?): Members of “backward” castes in rural Bihar are charged higher interest rates than members of other castes. The fact that credit relations are inherently social, and hence hierarchical and personalized, is not new or surprising, but the question of how meaning and hierarchy is embedded in money remains unexplored. Moreover, it begs the question of how an intervention like the Bihar Rural Livelihoods (JEEViKA) Project, by introducing a more egalitarian discourse around money, alters existing credit relations for lower caste women—both with respect to lenders and within their household.

(what?): This research seeks to examine how JEEViKA alters the social meaning of money. It explores how accessing a loan shifts the dynamics of dependency within the informal credit market and how exercising control over money increases respect for and dignity of women within the household.

(how?): Extensive information was collected through observation of more than 30 Self-Help Groups, village organizations, and cluster-level federation meetings, with a focus on the ways in which savings are handled and the language used to make an argument for a loan. This information was complemented by data extracted from the larger qualitative data set of 2,000 SHG members and nonmembers on what money and having control over money means to them. The study interviewed 400 husbands and other male family members of female SHG members, in order to gain perspective on the impact on the household of loans secured by women. In-depth interviews with different types of money lenders—ranging from large professional moneylenders to “amateur” lenders who lend money as a part-time activity—were conducted.

(findings): Preliminary analysis of the data suggests that the savings and

earnings aspect of group membership empowers women through multiple pathways. Within a SHG, learning to stake a claim over money and making a good argument to receive a loan fundamentally alters the notion that men alone can control money. It has empowering effects within both the SHG and households, where the securing of a loan gives women claims to haq (rights) over household finances and major household decisions. Accessing a loan bestows respect on women within the larger community. JEEViKA has reduced dependency on informal sources of credit, such as traditional moneylenders, employers, relatives, and friends. This impact is felt most strongly by women from “backward” castes, for whom the informal credit

24

market has always been more costly. Even the few women who continue to borrow from the informal market are better able to negotiate interest rates and terms of loans, considerably altering the dynamics of informal credit markets.

(project status): First cycle started: September 2011 Final cycle completed: March 2015

(outputs): Forthcoming

(principal investigators): Shruti Majumdar, Sociologist, Social Observatory, World Bank Vijayendra Rao, Head, Social Observatory, World Bank Paromita Sanyal, Assistant Professor of Sociology, Cornell University

What factors affect participation in community-driven projects? Implementation by facilitators → Increased agency of women + sustained

participation in project

(why?): Despite operating in a context of extreme poverty, deeply entrenched

inequalities, severe restrictions on women’s mobility, and an inherent mistrust toward development projects, JEEViKA elicited participation by thousands of women. How did it do so?

(what?): The Bihar Rural Livelihoods (JEEViKA) Project was not unconditionally welcomed at the village level at the outset, with husbands and powerful stakeholders in the villages particularly suspicious of the project. Despite this challenge, the vast majority of targeted households in both Phase I and Phase II villages were brought into the project, and in many cases their participation was sustained over time. One potential explanation for this change in perception and acceptance could be the process of “regimentation,” initiated by community resource persons but sustained over time by community mobilizers. Community organizers deal with both members of SHGs and their own employers (that is, the staff of JEEViKA). They are trained in the art and science of mobilization by ground-level JEEViKA staff, who themselves are carriers of complex structures of knowledge created by development bureaucrats in the capital and district offices. The community mobilizers are in essence the last link in a long “chain of translations.” They offer templates that condition the actions that Self-Help Group members take. Members of SHGs take the scripts of “empowerment”’ and “self-help” and adapt them based on their own experiences of poverty and poverty reduction. Focusing on the interface between these brokers and translators and the SHGs, where the repertoires are coproduced, can provide insight into the functions of a complex project like JEEViKA. This study examines the everyday practices of facilitation and mobilization to shed light on the functioning of JEEViKA.

25

(how?): The data for this study draw on the larger qualitative dataset of focus

group discussions and semi-structured interviews with more than 2,000 SHG members, nonmembers, and key stakeholders in eight villages collected over three years. In addition, the study collected the following data: (a) material used for trainings in JEEViKA (flipcharts, images, narratives), in order to understand the discourse of development as communicated from the capital office; (b) participant observation of the interface between trainers, facilitators, and SHG members, in order to capture the key moment of mobilization when the discourse “hits the road”; and (c) in-depth interviews with 50 community mobilizers, community resource persons, and SHG members in order to understand how they interpret and adapt the project language.

(project status): Data collected: September 2014; analysis ongoing

(outputs): Forthcoming

(principal investigators): Robindranath Banerji, Intern, Social Observatory, World Bank Shruti Majumdar, Sociologist, Social Observatory, World Bank Vijayendra Rao, Head, Social Observatory, World Bank Paromita Sanyal, Assistant Professor, Department of Sociology, Cornell University

A qualitative analysis of the processes by which JEEViKA shapes

gender relations in Bihar

Self-Help Groups → Improved agency + greater voice + increased empowerment?

(why?): Women and men are embedded in a larger ecosystem; if an

intervention affects women’s aspirations and capabilities at the individual level, it therefore also has a ripple effect on power dynamics and relational inequalities in the village as a whole. What is the impact of an intervention like the Bihar Rural Livelihoods (JEEViKA) Project on social change, caste, class, and gender dynamics at the village level?

(what?): The goal of this in-depth qualitative study is to understand the

processes underlying the impact of JEEViKA on women’s agency, voice, and empowerment. It examines the processes and mechanisms of change and the mediating influences that facilitate or obstruct it in order to determine whether there has been a reduction in social inequalities, an increase in the responsiveness of important public institutions, and a movement toward greater gender equity. The study seeks to identify the processes leading to poverty reduction, understand changes in women’s empowerment at the

26

household and community levels, and reveal the process and dynamics of social change. This study was preceded by a quantitative evaluation of panchayat blocks that joined JEEViKA in Phase I in 2006 using propensity scoring methodology methods. A separate randomized trial of JEEViKA’s impact in Phase II blocks is ongoing. This in-depth study of a much smaller sample of 11 villages is designed as a sociologically nuanced complement to these ongoing quantitative impact evaluations. The aim is to uncover the processes and dynamics of change for disadvantaged populations, a topic that is difficult to capture in quantitative impact assessments.

(how?): In each of the sample villages, a baseline study was conducted to obtain a comprehensive understanding of the layout of the village (through transect walks, key informant interviews, household mapping, and several participatory rural appraisal methods). Data were then collected in 12 cycles over four years (2011–15). Every three or four months, a team of four researchers from a Patna-based research organization (Praxis) collected data in 10 villages. By using a combination of qualitative methods, researchers collected data through conversations and in-depth unstructured interviews with program participants and nonparticipants observation of group meetings; interactions at the village, block, and district levels; focus group discussions with participants; and structured interviews with JEEViKA staff at all levels in all villages. Interviews, observations, and discussions were guided by a set of themes that was modified over the course of the data collection. The observations were conducted in the local language by researchers who had a good grasp of the local context.

(project status):

First cycle started: September 2011 Final cycle completed: March 2015

(outputs): Forthcoming

Principal investigators:

Shruti Mazumdar, Sociologist, Social Observatory, World Bank Vijayendra Rao, Head, Social Observatory, World Bank Paromita Sanyal, Assistant Professor, Department of Sociology, Cornell University

27

Can a domestic violence intervention change norms around violence and reduce its occurrence? Community support group formation + collective pledges → Reduced social

tolerance for violence

(why?): A violent act is committed against a woman every three minutes in India, according to a 2012 report by the National Crime Records Bureau (NRCB). One major form of this violence is intimate partner violence, a complex and deep-rooted issue that is entrenched in the patriarchy and gender norms of Indian society. Tamil Nadu reports the highest number of cases of domestic violence in India. In 2012, 3,838 cases of domestic violence were recorded in the NCRB database—an astonishing 84 percent of all complaints of domestic violence in India. Although these figures may partly reflect greater awareness and better reporting in Tamil Nadu than elsewhere, a problem does exist. Violence against a wife as a way of addressing domestic conflict is considered a social norm, a reality that a woman is expected to accept for the sake of her children and family. Even if a wife is able to support herself and her children, she is unlikely to do so because of the social stigma attached to separation and divorce. How can an effective domestic violence intervention be designed within this sociocultural context?

(what?): The Social Observatory is providing technical assistance to design, implement, monitor, and evaluate strategic communication interventions that attempt to modify norms surrounding the acceptability of intimate partner violence.

(how?): The program will evolve organically as the project is pre-piloted in sample areas. This evolution will ensure the development of practical and feasible approaches that can be scaled up in 2015 to 400 villages within the Tamil Nadu Empowerment and Poverty Reduction (Pudhu Vaazhvu) Project. Once the program intervention that emerges from this field-tested pilot is in place, the Social Observatory will use a randomized control trial to evaluate the effectiveness of different ways to address domestic violence.

(project status): Design: Expected March 2015

(outputs): Forthcoming

(principal investigator):

Vijayendra Rao, Head, Social Observatory, World Bank

(design and implementation support): Uttara Bharath Kumar, Senior Program Officer, Johns Hopkins Center for Communications Program

28

Varsha Ramakrishnan, Public Health Consultant, Social Observatory, World Bank

29

Bihar The Role of Quantity Discounts in Achieving Equitable Access to Food

Evaluating JEEViKA’s Food Security Fund

JEEViKA Retrospective Evaluation

JEEViKA Core Evaluation

Impact of JEEViKA: Altering the Social Meaning of Money

JEEViKA Facilitation Module

JEEViKA’s Impact on Gender Relations

Karnataka The Anatomy of Failure

Maharashtra WELLS Tracking Survey

Odisha Strengthening Livelihoods for Resilience to Disasters

Odisha Core Evaluation

WELLS Tracking Survey

Rajasthan Tamil Nadu Using Community-Based Approaches to Reduce Anemia

Tamil Nadu Retrospective Evaluation

Tamil Nadu Core Evaluation

Gram Sabha Recordings Evaluation

Women’s Participation Evaluation

Gram Panchayat Incentive Fund Evaluation

Domestic Violence