assessing interventions to improve child nutrition: a theory-based impact evaluation of the...

Journal of International Development

J. Int. Dev. 19, 627–652 (2007)

Published online 19 December 2006 in Wiley InterScience

(www.interscience.wiley.com) DOI: 10.1002/jid.1344

ASSESSING INTERVENTIONS TO IMPROVECHILD NUTRITION: A THEORY-BASED

IMPACT EVALUATION OF THEBANGLADESH INTEGRATED

NUTRITION PROJECTy

HOWARD WHITE* AND EDOARDO MASSET

Independent Evaluation Group, World Bank, Washington DC, USA

Abstract: The Community-Based Nutrition Component of the Bangladesh Integrated Nutri-

tion Project sought to improve nutritional status through nutritional counselling and supple-

mentary feeding for malnourished children and pregnant women. This paper presents a

theory-based impact evaluation of this programme. Both counselling and feeding suffered

from problems of inappropriate targeting strategies and a failure to reach intended groups.

While counselling has changed women’s knowledge it has had less of an impact on behaviour.

There is a knowledge-practice gap, which is explained by resource constraints faced by

women, including lack of time, which prevent them from putting the advice into practice. The

impact of the project on anthropometric outcomes of infants, children and mothers has not

been large. Copyright # 2006 John Wiley & Sons, Ltd.

Keywords: food nutrition; Bangladesh; behaviour change

1. INTRODUCTION

In the 1980s malnutrition levels in South Asia were the highest in the world and those in

Bangladesh were the worst in the region. Stunting affected two-thirds of Bangladeshi

*Correspondence to: H. White, World Bank, 1818 H Street, Washington DC, 20433 USA.E-mail: [email protected] paper is based on work undertaken for an impact evaluation of interventions to improve maternal and childhealth and nutrition in Bangladesh (World Bank, 2005a), which was carried out under an IEG-DFID partnershipagreement. The paper also draws on material from IEG’s Project Performance Assessment Report of the project(World Bank, 2005b). Many people, too numerous to list here, assisted with those studies. But acknowledgement isduly given to the other members of the study team, Nina Blondal and Hugh Waddington, and to Meera Shekar(World Bank), Dora Panegides (Helen Keller International, Dhaka), and Anna Taylor and Arabella Duffield (Savethe Children, UK) for facilitating access to their respective data sets. The findings, interpretations, and conclusionsexpressed in this article do not necessarily reflect the views of the Executive Directors of the World Bank or thegovernments they represent, nor those of DFID.

Copyright # 2006 John Wiley & Sons, Ltd.

Table 1. Anthropometric outcomes in South Asia (percentage of children below-2 standarddeviations from the reference median)

1975–79 1980–84 1985–89 1990–94 1995–99 2000–04

Stunting (HAZ)

Bangladesh — 67.7 67.5 64.4 53.9 44.7

India 72.3 — — 58.4 44.9 —

Sri Lanka 44.6 36.2 27.2 23.8 20.4 —

Pakistan 67.0 — 57.9 42.9 — —

Underweight (WAZ)

Bangladesh — 68.0 70.9 67.1 59.2 47.7

India 71.3 — — 59.4 46.7 —

Sri Lanka 54.3 — 37.3 37.7 32.9 33.0

Source: World Bank, World Development Indicators.

628 H. White and E. Masset

children under 5 years in the early 1980s, a higher proportion than in both India and

Pakistan, and far higher than in Sri Lanka (Table 1). This gap had widened by the early

1990s, as there had been virtually no improvement in Bangladesh, whereas malnutrition

rates were improving in the other countries in the region.

This dismal trend in Bangladesh in part reflects a lack of serious policy attention to

nutrition, standing in stark contrast to the country’s success in reducing fertility through an

extensive family planning programme initiated in the 1970s, and the acceleration of mortality

decline assisted by increased immunisation in the 1980s. The Bangladesh National Nutrition

Program was set up in 1975 but carried out only a limited range of activities. It was against

this background that the World Bank proposed the Bangladesh Integrated Nutrition Project

(BINP), to be modelled on the Tamil Nadu Integrated Nutrition Project (TINP).

TINP was a World Bank-supported project implemented between 1980 and 1989, with

the objective of improving the nutritional and health status of pre-school children, pregnant

women and nursing mothers. The intervention consisted of a package of services including:

nutrition education, primary health care and supplementary feeding, and was the first

project to employ Growth Monitoring and Promotion (GMP) on a large scale.1 The GMP

approach sees food as a ‘medicine’ (Berg, 1987), which is provided at community health

centres, where children are weighed by community workers in order to detect

malnourishment. These community workers also instruct women on how to make better

use of the existing resources in feeding their children. This approach mirrored

developments in the economic literature, including research at the World Bank, arguing

that income-based improvements in nutrition would be inadequate, so there is a strong case

for nutrition education. TINP has been widely held up as a success story, being called one

of the ‘most successful [projects] in the world in reducing malnutrition’.2 An impact study

of TINP, conducted by the Operations Evaluation Department (OED)3 of the World Bank

in 1994, found that the project achieved unprecedented rates of decline in malnutrition in

project areas, most of which was attributable to the project (World Bank, 1994). GMP has

since become the dominant approach to nutrition interventions, though some suggest the

evidence base for this approach is weak; for example, in their review of such programmes,

1Details of the project and an evaluation of its impact on the nutritional status of children can be found in WorldBank (1994).2http://www.worldbank.org/ourdream/india_2.htm (accessed on 04/14/05).3OED was renamed the Independent Evaluation Group (IEG) at the end of 2005. The name IEG is used for currentwork, including the study discussed in this paper, but OED retained for older studies.

Copyright # 2006 John Wiley & Sons, Ltd. J. Int. Dev. 19, 627–652 (2007)

DOI: 10.1002/jid

Assessing Interventions To Improve Child Nutrition 629

Panpanich and Garner (1999) state that ‘given the level of investment in growth monitoring

worldwide, it is surprising there is so little research evaluating its potential benefits and

harms’.

This paper presents an analysis of the Bangladesh Integrated Nutrition Project. This

project, which was subject to controversy at the design stage (Save the Children, 2003;

World Bank, 2005b) has again become the centre of debate largely prompted by a critical

report by Save the Children (2003), summarised in Hossain et al., (2005), whose findings

have been contrasted with those of the evaluation contracted by the project (Karim et al.,

2003, referred to here as ‘the BINP evaluation’). This debate has become more heated

given the decision to replicate BINP nationally under the National Nutrition Project (NNP).

The study carried out by IEG, which is summarised in this paper, brought two innovations

to this debate. First, the evaluation design was explicitly rooted in a theory-based approach

(theory-based evaluation, TBE), intended to test the links in the causal chain (or logic model)

from inputs to impact; see Weiss (1998) for a general exposition of this approach, and

Carvalho and White (2004) for an application in a developing country context. The

underlying logic of the intervention is laid out in the next section, which provides an

introduction to the project; it is this logic that TBE seeks to test. These tests were carried out

drawing on a range of data, including re-analysis of the primary data from both the studies

mentioned in the previous paragraph. Second, more rigorous quantitative techniques were

employed, utilising the BINP evaluation project area data but constructing a new control

group using propensity score matching with data from the nationwide Nutritional

Surveillance Project. ‘Register data’ (i.e. the growth monitoring records) which were

captured by Save the Children in their project sample villages, were also analysed.

These methods and the data employed are presented in Section 3 of the paper. The remainder

of the paper focuses on findings: project coverage and targeting (Section 4), changes in

knowledge and practice (Section 5) and nutritional impact (Section 6). Section 7 concludes.

2. BINP: OVERVIEWAND UNDERLYING THEORY

2.1. The Project

BINP was launched in 1995 with the objective of ‘reducing malnutrition to the extent that it

ceases to be a health problem’ (World Bank, 1995). The project was expected to cover 59

thanas in three phases during 5 years of implementation.4 The project had three

components: (1) National Nutrition Activities; (2) Community-based Nutrition (CNBC);

and (3) the Inter-sectoral Nutrition Program Development, supporting schemes, such as

home gardening and poultry rearing intended to have beneficial effects for nutrition.

The main component, CNBC, which is the one focused on here,5 was based on growth

monitoring, food supplementation and nutritional counselling. Under BINP, CNBC was

phased in over time, operating in 59 rural thanas across the country by the close of the

project, accounting for approximately 12 per cent of the country’s population. Nutritional

4Thana (also called Upazila) is an administrative unit (sub-district), of which there are a total of 496 inBangladesh, each comprising an average of 250 000 inhabitants.5CNBC had the largest budget share, in addition to which many activities under the national component, such assupport to the BINP project office, were indirectly also for the implementation of CNBC. The inter-sectoralcomponent was conducted in CNBC villages only, and the limited evidence regarding its effectiveness ismentioned in the paper.


DOI: 10.1002/jid


counselling was at the heart of CNBC, with the other activities providing a framework for

such counselling. Community Nutrition Promoters (CNPs) were recruited in each

community at a ratio of one CNP for every 1000 population in some areas and every

1500 in others. CNPs, recruited from women with children, and a minimum 8 years of

education, were responsible for implementation of most project activities at the community

level, in collaboration with the Women’s Group; they worked under the supervision of the

Community Nutrition Organizer (CNO, each of whom was responsible for 10 CNPs). The

work of the CNP was supported by the Village Nutrition Management Committee (VNMC).

The programme adopted two different target audiences, the second of which was a pilot

in just one of the six thanas under the first phase of the project. Under the approach used in

most thanas, all children aged under 2 years were intended to be covered by monthly

growth monitoring, with feeding supplementation offered to severely malnourished and

growth faltering children under 2 years (meeting criteria based on weight-for-age charts)

and malnourished pregnant women (meeting criteria based on midarm circumference).

Under the experimental targeting strategy in one thana all newly married couples in the

project area were provided a special package of health and welfare services until the first

child reached 2 years old.6

The programme was implemented in two different ways. Some thanas were wholly

contracted out to non-governmental organisations (NGOs) for the management and

implementation of all CBNC activities. In the other thanas, the Government of Bangladesh

(GOB) used its own management structure to run the project activities; NGO support was

provided in selected key areas (community mobilisation, training and technical supervision

of field personnel, and the logistics of preparing food supplements).

2.2. The Theory Underlying the CNBC

The theory-based approach breaks down the logic underlying the intervention, tracing the

steps from inputs to impact. Even a simple intervention, such as supplying supplementary

feeding to reduce malnutrition, contains a number of, often implicit, assumptions, which

have to be met for the input to have the desired impact. In the case of BINP, there are two

associated logic models, one related to changing practice and the other to supplementary

feeding. The challenge for the evaluation is to test whether the expectations linked to these

assumptions were realised in practice.

The key assumption behind CNBC is that ‘bad practices’ are responsible for

malnutrition in Bangladesh. This point of view was strongly argued in the BINP appraisal

document: ‘behaviours related to feeding of young children have at least as much (if not

more) to do with the serious problem of malnutrition in Bangladesh as poverty and the

resultant household food insecurity do’ (World Bank, 1995: 4). Therefore changing bad

practice to good will bring about nutritional improvements. There are a number of steps in

the causal chain behind this approach:

1. T

6Thdiffwouthe

Cop

he right people (those making decisions regarding under-nourished children) are

targeted with nutritional messages

e BINP evaluation data set covers the six thanas from the first phase. Whilst the intention was to test theerent approaches this idea was poorly thought out, since the ‘couples approach’ was used in just one thana itld not be possible to separate the area effect from the project design effect. The problem was exacerbated byfact that there were other variations in design (e.g. population per CNP and implementing agency).

yright # 2006 John Wiley & Sons, Ltd. J. Int. Dev. 19, 627–652 (2007)

DOI: 10.1002/jid


2. T

7Thundsem8Thdetecen

Cop

hese people participate in project activities, and so are exposed to these messages

3. E
xposure leads to acquisition of the desired knowledge
4. A
cquisition of the knowledge leads to its adoption (i.e. a change in practice)
5. T
he new practices make a substantial impact on nutritional outcomes
Supplementary feeding was provided to malnourished children and pregnant women.

For this programme to work:

1. T
he target groups have to enroll in the programme
2. T
he criteria are correctly applied in selecting those to receive supplementary feeding
3. T
hose selected for supplementary feeding attend sessions to receive the food.
4. T
here is no leakage (e.g. selling of food supplements or giving food to other family
members), or substitution (reducing other food intake)

5. T
he food is of sufficient quantity and quality to have a noticeable impact on nutritional
status (and, in the case of pregnant women, to be of sufficient magnitude to have an

appreciable impact on birth weight).

If project design fails to take account of one of these steps, there is a missing link in the

causal chain. If activities take place corresponding to a step but they are ineffective, then

there is a weak link. We next discuss the data and methods used to analyse these links.

3. DATA AND METHODS

3.1. The Three Data Sets

This paper uses data from three different sources, the BINP evaluation data, the Save the

Children evaluation data, and the Helen Keller International (HKI) nutrition data. Each dataset

covers different aspects of the nutrition intervention, and none of them taken alone would allow

the comprehensive analysis of the project carried out in the paper. Taken alone each dataset has

inadequate coverage of all relevant topics to carry out a full theory-based impact evaluation. In

addition, Participatory Practitioner’s Society-Bangladesh (PPS-BD) conducted a qualitative

study in June 2004 to examine the existence of traditional beliefs related to child and maternal

malnutrition, reasons for mothers’ inability to participate in project activities and transform the

acquired health knowledge into practice (World Bank, 2005a).7

The BINP evaluation conducted three cross-sectional surveys of children, pregnant

mothers and infants in September–October 1995 (baseline), November–December 1998

(midterm), and January–March 2003 (end-line). The survey sampled households in six

first-phase ‘project’ thanas and two matched ‘control’ thanas.8 The questionnaires

collected information on demography, social and economic variables, nutritional status of

children and their mothers, nutritional practices, pregnancy weight gain and birth weight.

For reasons of data quality, only the midterm and the end-lines samples are used in the

analysis. The BINP data were used to identify the determinant of mothers’ participation in

e qualitative study was commissioned by the IEG. The research was conducted in two communities fallinger BINP and two in non-BINP villages. In addition to focus groups and key informant interviews,i-structured interviews were conducted with 150 individuals.e samples were calculated using standard formula for unequal sample sizes. The samples were designed toct a difference of the order of 3 per cent in malnutrition rates at 95 per cent significance and a power of 80 per

t. A two-stage cluster sampling was employed, but the design effect was considered to be zero.


DOI: 10.1002/jid


project activities, the determinants of infants’ birth weight, and malnutrition rates of

children aged 6–24 months.

The Save the Children sample consists of a single cross-section survey of children and

mothers, conducted between March and September 2002. The survey sampled households

from three ‘project’ and three ‘control’ thanas.9 The questionnaires collected information on

household demographic and socio-economic characteristics, child illnesses, maternal

knowledge and caring practices. We used these data in order to estimate mothers’

‘knowledge-practice gap’. Save the Children also collected data of children enrolled in

supplementary feeding from the project registers. During the survey 1450 children aged 6–59

months were found who had been previously enrolled in supplementary feeding. Records of

these children were tracked in 70 per cent of cases. Register data included age at entrance in

the programme, weight for each month, and the amount of food provided. These data were

used to estimate the determinants of weight gain during supplementary feeding.

The Nutritional Surveillance Project (NSP) of HKI collects nutrition data that are

representative at the division and national level. Every 2 months, some 10 000 households

are sampled from 24 randomly selected thanas around the country. A household is eligible

for inclusion if it contains at least one child aged less than 5 years, and if the mother is

present. The dataset contains data on location variables, household and child

characteristics, and anthropometric measurements. This sample was used to construct a

control group, using propensity score matching methods (described below), in order to

estimate project effects on child nutritional status.

3.2. Econometric Methods

Econometric estimates are made for various links of the causal chain: participation,

acquiring and utilising knowledge and nutritional outcomes. The models are OLS, probit

and logit as appropriate. Here we elaborate on the less familiar method of propensity score

matching which was used to assess the project impact on child nutritional status.

Estimates of project effects based on simple comparisons of means between project and

control areas were found to be poor on two accounts. First, the size and the quality of the

BINP and the Save the Children control group samples were inadequate. Second,

simple comparisons of means ignore the selection bias deriving from the fact that not

all sample children are participating in the project, and that participating and

non-participating children are likely to have different characteristics. The evaluation

problem consists of estimating the project effect T (‘treatment effect’) expressed as the

difference in the outcome Yip (the nutritional status of children) children i with and without

participation:

T ¼ E Yi1 � Yi0½ � (1)

Since participation is on a voluntary basis, and while some mothers participate in the

project (D¼ 1), others do not, the project effect for the participants (‘treatment effect on the

treated’) is:

Tt ¼ E Yi1 � Yi0 D ¼ 1j½ � (2)

9Sample sizes were calculated using an inverse sampling procedure. A two-stage cluster sampling was employed,and the design effect for the correction of sample sizes was estimated at 1.2.


DOI: 10.1002/jid


The problem in calculating (1) and (2) is that we do not observe the outcome Yi for the

same child with and without the project. In order to estimate the project effect we need to

construct a comparison group whose characteristics are identical to the characteristics of

the project group before project implementation. This comparison group is then used to

calculate the outcome that project participants would have experienced without the project.

We construct the comparison group using propensity score matching (PSM). Ideally we

would like to match each ‘project observation’ with a ‘control observation’ which is

identical in all relevant respects, e.g. education, wealth, age and so on. Clearly it is difficult,

if not impossible, to find such matches based on multiple criteria. However, Rosembaum

and Rubin (1983) have shown that we get the same outcome, if we match, based on a linear

combination of these characteristics; more formally equation (2) is equivalent to:

Tt ¼ E Yi1 � Yi0 eðxÞj½ � (3)

where, e(x)¼ Pr(D¼ 1jX), is the probability of participation given observed individual

characteristics. This result shows that matching observations based on their characteristics

is equivalent to matching the observations based on the predicted participation in the

project based on the same characteristics. The probability of participation e(x) is estimated

using a model of qualitative choice (e.g. probit), and the resulting predicted probabilities

(‘propensity scores’, hence the name PSM) are used for the matching of observations

instead of the characteristics themselves. The control group used in the propensity score

matching analysis was created using the NSP sample. Observations were obtained for the

same months in which the BINP survey had been implemented in order to avoid differences

in nutritional outcomes determined by seasonal fluctuations.

The probability of participation was estimated using 35 explanatory variables. The

t-tests on the equality of the means of the explanatory variables in the project and in the

non-project groups after matching showed no significant differences at 5 per cent level; i.e.

the project and control groups are ‘the same’. Observations outside of the region of

‘common support’ (that is observations with probabilities of participation either less or

greater than any in the other group) were dropped, and the effects were calculated using

three different methods, one-to-one matching, nearest neighbour matching and kernel

matching.

4. PROJECT COVERAGE AND TARGETING

BINP nutritional counselling activities target pregnant and lactating women and adolescent

girls.10 However, strategies to affect health behaviour need to influence the attitudes of all

those involved in making health decisions. In Bangladesh decisions regarding health and

nutrition do not rest solely with the mother, but also the husband and frequently the

mother-in-law. Data from the Demographic and Health Survey (DHS) show that only one

in five women are solely responsible for decisions regarding their own health and that of

their children, falling to only 1 in 10 of women living with their mother-in-law (Table 2).

While women are most likely to have responsibility over deciding what to cook

(two-thirds of women are solely responsible for this decision, though this is so for only

10In addition to the adolescent girls’ forum, there is a married couples’ forum. However, during fieldwork, onlywives were found to be present at such fora.


DOI: 10.1002/jid

Table 2. Women’s say in decision-making by position in household (percentages)

Own health Child’s health Daily purchases Cooking

All respondents (n¼ 9716)

Respondent 20.6 19.2 19.5 66.8

Joint 36.1 47.1 43.8 20.4

Husband 36.9 26.9 27.2 4.6

Someone else 6.4 6.9 9.6 8.2

Currently married women in male-headed households (n¼ 8706)

Respondent 15.7 14.3 15.5 62.8

Joint 36.8 50.0 46.1 19.3

Husband 41.0 29.9 30.2 8.7

Someone else 5.1 5.8 8.2 9.2

Living with mother in law (n¼ 2017)

Respondent 12.2 10.3 9.8 42.8

Joint 36.2 48.5 45.2 31.3

Husband 39.5 27.1 22.8 3.3

Someone else 12.2 14.5 22.2 22.6

Note: The sample for DHS is ever-married women aged 10–55.Source: Calculated from 1999/2000 DHS data.


42 per cent of women living with their mother-in-law), which clearly matters for child

nutrition, the effectiveness of this decision-making power is constrained by the fact that it is

men who do the shopping in Bangladesh. This fact is reflected by the fact that only 17 per

cent of married women in male-headed households are responsible for decisions about

daily purchases, and only 10 per cent of those women living with their mother-in-law have

this responsibility. Hence, the project’s nutritional messages appear to have been too

narrowly focused.11

Ideally all children in the project area should participate in growth monitoring, with a

target set in the project appraisal document that 80 per cent of 6–24 month olds should be

registered, and 80 per cent of these (so 64 per cent of all children) receive at least 18 out of

24 monthly weighings. The BINP evaluation data show that over 90 per cent of children

were weighed at least sometimes, with 88 per cent being weighed on a regular basis.12

Hence the project’s coverage targets were met, although these allowed for some children to

go unmonitored. If the children not attending have little risk of malnutrition then their

omission can be seen as an efficiency gain for the project rather than a shortcoming. The

opposite is true if not attending children are those at higher risk of malnutrition. Studies of

GMP programmes have sometimes found lower participation by economically

disadvantaged mothers.13

Multivariate analysis of participation was carried out with both a logit (participant¼ 1)

and multinomial logit (non-participant¼ 0; occasional participant¼ 1 and regular

participant¼ 2) models using the BINP evaluation midterm data.14 After estimation of

11The BINP appraisal document makes some mention of other decision makers, but the main focus of both designand implementation has been mothers.12Calculated from end line data using responses ‘Always’ or ‘Almost always’ attend.13See, for example, Chopra and Sanders (1997), and the review of efficacy of GMP programmes conducted byPearson (1995).14Total sample size of mothers with children aged from 6 to 23 months, and therefore eligible for growthmonitoring and promotion, was 3662. Some observations had to be dropped because of missing data.


DOI: 10.1002/jid


the multinomial model, we tested the null hypothesis that any pair of the three categories

could be combined; this test is a likelihood ratio test of the hypothesis that all coefficients of

the multinomial logit model associated with any given pair of outcomes were zero. Both

the hypotheses of equivalence of non-participation and occasional participation

(chi-square of 57.8 and p-value of 0.000), and of occasional and regular participation

(chi-square of 117.9 and p-value of 0.000) were rejected. The hypothesis of independence

of other alternatives was also not rejected by a Hausman test, thus making unnecessary the

use of more complex estimation models.

The results for midterm and end line were similar and only those for the midterm are

shown here (Table 3). Figure 1 illustrates the probabilities of participation for mothers with

specific characteristics. The main conclusions are that:

1. T

Var

Chi

Chi

Chi

Mo

Fem

Mo

Dau

Dau

R

Wid

Num

Prim

Sec

Hig

Hou

No

No

Far

Gab

Mo

Raj

Con

Pse

Obs

Cop

here is no sex bias in participation. Participation has a non-linear relationship with the

child’s age, with those aged 9–10 months most likely to attend.

2. B
oth younger mothers and daughters-in-law are less likely to attend (daughter-in-law
meaning here a woman living with her mother-in-law). The daughter-in-law dummy

alone is significantly negative (results not shown), but is no longer so when an

interactive term is included for the daughter-in-law dummy and the dummy for the

Table 3. Determinants of participation in growth monitoring (midterm BINP data)

iable Logit Multinomial logit

Participants Occasionalparticipants

Regularparticipants

Coeffecient t-statistic Coeffecient t-statistic Coeffecient t-statistic

ld sex �0.02 �0.12 0.12 0.59 �0.04 �0.26

ld age 0.19 1.77� 0.20 1.62� 0.20 1.79�

ld age squared �0.01 �1.85� �0.01 �1.39 �0.01 �1.92�

ther’s age �0.04 �3.08�� 0.05 �2.60�� 0.04 �3.04��

ale headed household 0.00 �0.01 �0.21 �0.39 0.04 0.08

ther works 1.34 1.23 0.15 0.11 1.45 1.33

ghter-in-law �0.19 �0.54 �0.15 �0.36 �0.20 �0.56

ghter-in-law in

ajnagar and Shahrasti

�1.13 �2.15�� 0.81 �1.37 �1.19 �2.22��

ow �0.45 �0.49 0.04 0.04 �0.56 �0.61

ber of pregnancies 0.18 1.63� 0.01 0.09 0.21 1.82�

ary education 0.13 0.52 0.05 0.19 0.14 0.57

ondary education 0.42 1.35 0.27 0.79 0.45 1.42

her education �0.41 �1.64� �0.49 �1.41 �0.40 �1.65�

se index �0.22 �2.46�� 0.35 �3.26�� 0.20 �2.20��

drinking water �0.98 �2.12�� 0.63 �1.30 �1.08 �2.23��

latrine �0.01 �0.04 �0.10 �0.34 0.00 0.01

idpur �1.64 �3.91�� 0.32 �0.74 �1.76 �4.03��

toli �0.21 �0.35 0.23 0.48 �0.23 �0.38

hammadpur �0.76 �1.84� 0.19 0.43 �0.82 �1.96��

nagar and Shahrasti �1.21 �2.96�� 0.08 0.19 �1.32 �3.20��

stant 4.55 4.58�� 2.07 1.84� 4.38 4.35��

udo R-square 0.12 0.07

ervations 3541 3541


DOI: 10.1002/jid

Figure 1. Participation in children growth monitoring session by mothers’ characteristics.

15It1216‘N17TIncsouwatand18Hgoo

Cop


two more conservative thanas, Rajnagar and Sharasti.15 The significant negative

coefficient on this interactive term shows the effect of restrictions on female mobility

in attending growth monitoring sessions, although they only have an effect in two of the

six thanas.

3. Q
ualitative analysis shows that remoteness (travel time) is also a constraint on
participation. There is no direct measure of this variable in the dataset, but it is likely

that type of water supply (‘no drinking water’16) is a reasonable proxy for this variable

(it is hard to imagine a direct effect of this variable), the results confirming that inferior

forms of water supply (indicating remoteness) significantly reduce participation.17 The

results confirm that highly educated women are less likely to participate, as are those

from better-off households (proxied by housing quality). Since both these variables are

correlated to child nutrition, the results suggest that the children of those not participat-

ing are likely outside the target group, except those from remote communities.

The project addressed low child growth, identified by growth monitoring, in two ways:

nutritional counselling and supplementary feeding. The feeding was intended as an

example to mothers, the heart of the strategy being counselling to achieve behaviour

change.18 Field observations suggest that growth monitoring sessions themselves were too

chaotic a setting to provide nutritional counselling. But CNPs work full time in their

is also more common for a woman to be living with her mother-in-law in these thanas, being so for 15 andper cent of respondents in Rajnagar and Sharasti, respectively, compared to 9 per cent on average.o drinking water’ means households rely on rainwater, rivers and ponds for domestic use.

he BINP dataset does not contain data on distances to health facilities. Using data of the Bangladesh Householdome and Expenditure Survey (HIES) of 2000–01, it was found that the time distance to the health providerght for treatment was substantially longer for household not having access to drinking water (i.e. accessinger from rivers, ponds, rain etc.). Average travel time was calculated in 27 and 40 minutes for households withwithout water access, respectively (t-stat¼ 2.72, p-value¼ 0.006).

owever, the same supplement was provided throughout supplementary feeding which does not appear to be ad example regarding nutritional practices.


DOI: 10.1002/jid


position and provide advice through different fora, such as the various group counselling

sessions or one-to-one meetings with parents. However, over one-third of women whose

children were receiving supplementary feeding said that they had neither discussed

nutrition directly with the CNP, nor sat in any meeting where it was being discussed

(source: Save the Children data).

Supplementary feeding was provided to children who were detected to be severely

malnourished (less than �4 standard deviations from the reference median weight for age)

or experiencing growth faltering19 for a period of 3 months, extended for a further month if

they did not show the required weight gain. Analysis of register and field data collected by

researchers from Cambridge (Mascie-Taylor, 2004) showed a relatively low Type II

targeting error:20 only 16 per cent of children receiving food supplementation should not

have been doing so.21 However, of those enrolled, only one-quarter (26 per cent) received

supplementation for the recommended 3 months, the majority dropping out sooner. And

Type I targeting error was very high: over two-thirds (69.8 per cent) of eligible children

were not being fed.

Whilst quite a low percentage of ineligible children were being fed, the criteria

themselves are open to question. Growth faltering is quite normal, so enrolling growth

faltering children regardless of their nutritional level will mean enrolling perfectly

well-nourished children: one study found that 37 per cent of a group of US children from

well off backgrounds would qualify for supplementary feeding under the criteria used in

TINP (Martorell and Shekar, 1992). The Save the Children register data show that over 40

per cent of those enrolled in the feeding programme were not malnourished, i.e. the

children were above �2 standard deviation of the reference weight-for-age (WAZ).

Turning to monitoring of pregnancy weight gain, close to three-quarters of pregnant

women attended weighing sessions, and just under half received supplementary feeding.

There is no pattern between attendance at weighing sessions and the mother’s nutritional

status, which is to be expected. Supplementary food was meant to be received by women

not having a normal Body Mass Index (BMI). The cut-off point used to detect

pre-pregnancy malnutrition was set at a BMI of 18.5. However, by the end line about 60 per

cent of eligible women were not receiving the supplement. On the other hand, 40 per cent

of those who were receiving the supplement were not eligible.22

For both children and mothers there is evidence of both leakage and substitution of the

food supplement. For example, 32 per cent of women said they had shared their food

supplement with someone else. Many of those who were not sharing said they did not eat

more during pregnancy than usual, indicating that the BINP provided food was substituting

19During fieldwork differing criteria appear to have been applied to detecting growth faltering, which may in partreflect poor understanding by CNPs (confirmed by tests of 20 CNPs carried out by IEG) or confusion arising fromthe transition to new criteria under NNP. For normal children, the official criteria appear to have been less than600 g gain in 2 months for those aged 6–12 months and 300 g in 2 months for those aged 12–24 months. Forchildren with WAZ<�2 SDs these figures apply to a 3 month period.20Type I targeting error is the proportion of the target group not receiving the benefit, and Type II targeting errorthe proportion of those receiving the benefit who are not eligible to do so.21This analysis requires information on pre-feeding nutritional status, which is not available in the BINPevaluation and Save the Children datasets. Save the Children also collected register data, but only for children inthe feeding programme, so Type I error cannot be calculated from these data. The Type II error calculated from theSave the Children register data is 15.4 per cent, almost identical to that found in the Cambridge studies.22Data are not available on pre-pregnancy weight. The BMI used here is calculated from data for women not morethan 6 months pregnant, so there will have been some pregnancy weight gain, thus providing an upward bias to theestimate of both Type I and II error, but not to such a great extent as to negate the argument being made here.


DOI: 10.1002/jid

Table 4. Knowledge of nutrition practices in project and control areas compared (percentages)

BINP (end-line) Save the Children

Project Control t-statistic Project Control t-statistic

Rest during pregnancy 89.0 58.0 23.4�� 77.1 69.4 4.0��

Colostrum 97.4 92.4 4.9�� 62.7 51.5 4.3��

Breastfeeding 96.6 89.6 5.9�� 77.6 68.7 5.1��

Note: �significant at 10%, ��significant at 5%, ��significant at 1%.


for other foodstuffs. This was possible, since many women and children, contrary to project

design, consumed the food at home.

In summary, enrollment in growth monitoring sessions has been at a high level for both

children and pregnant women. However, attendance at these sessions has not provided

opportunities for nutritional counselling for a sizeable minority of women. There have been

problems in the targeting of feeding programmes, especially the exclusion of eligible

participants for both children and pregnant women. In the case of pregnant women, a

considerable number of feeding beneficiaries are in fact ineligible. Such Type II targeting

error is less of a problem for child feeding, though the entry criteria are inappropriate so

many well-nourished children are enrolled, and only a minority of enrolled children

complete the full 3 months of feeding.

5. NUTRITIONAL KNOWLEDGE AND PRACTICE

5.1. Acquiring Knowledge

The central thrust of project design was to change nutritional behaviour of child caretakers.

There are a number of nutritional practices considered adverse to child nutrition. Some are

simple differences in habit, such as cutting vegetables before washing them rather than vice

versa, which is nutritionally disadvantageous. Others, such as eating less during pregnancy

(‘eating down’), result from different perceptions of health risks and benefits (mothers

perceive the benefit of a lower risk delivery of a smaller child,23 discounting the risks to low

birth weight children). And others stem from traditional beliefs, which appear to have no

plausible health-related rationale, such as avoiding meat, fish and eggs during pregnancy.

Data from both Save the Children and the BINP evaluation show that knowledge with

respect to good nutritional practices is indeed higher in project areas than in the control.

Table 4 show percentages of women aware of selected nutrition practices based on the

BINP and the Save the Children data. Percentages vary largely between the two samples,

reflecting the different ways in which the questions were phrased in the two surveys.

However, both sets of data show a higher, and statistically significant, knowledge of

nutrition practices in project areas.

The BINP dataset also allows examination of the double difference for the knowledge of

resting and eating more during pregnancy (i.e. of the change over time compared between

23Eating down was recommended medical practice in European countries for this reason until the 1940s. There isnot unanimity regarding the ‘accepted wisdom’, with a recent Australian study finding that women deliveringchildren with a higher birth weight are more likely to require a Caesarian section.


DOI: 10.1002/jid


project and control areas). For both indicators there has been an increase in knowledge in

the project area but not in the control. At the baseline, a slightly higher percentage of

women in the control thought it advisable to eat more during pregnancy than was the case

in the project area. This proportion had not changed in the control areas, whereas in the

project area it rose by 28 per cent (which is therefore the double difference effect, since

there was no change in the control).

Of course other factors may be affecting knowledge, such as superior education in the

project area. A multivariate probit model was estimated to control for these factors (the

knowledge equation in the top half of Table 5). The probability of a mother knowing a

Table 5. Knowledge and KP gap equations

Rest during pregnancy Colostrum Breast-feeding


Knowledge (selection) equation

Mother’s age 0.04 2.27�� 0.01 �0.50 0.03 1.49

Mother’s age squared 0.00 �2.18�� 0.00 0.01 0.00 �2.13��

Mother primary education 0.26 3.59�� 0.18 2.51�� 0.20 2.29��

Mother secondary education 0.44 4.94�� 0.43 5.20�� 0.54 5.43��

Mother tertiary education 0.36 1.97�� 0.98 5.80�� 0.91 4.61��

Education h/h head 0.02 3.00�� 0.02 3.04�� 0.01 2.07��

Female headed household 0.33 3.04�� 0.22 2.28�� 0.41 3.90��

Group member 0.07 1.92� 0.07 2.04�� 0.08 1.86�

Project area 0.16 2.16�� 0.21 2.44�� 0.22 2.94��

Supplementary feeding �0.07 �0.95 0.01 0.13 �0.02 �0.26

Attending nutrion meeting 0.27 3.35�� 0.23 3.24�� 0.19 2.08��

Partner in nutrition discussions 0.13 1.68� 0.12 1.45 0.01 0.16

Project� primary education 0.03 0.28 0.00 �0.02 0.10 0.93

Project� secondary education 0.05 0.47 0.11 1.05 0.07 0.58

Project� tertiary education 0.40 1.75� 0.11 0.49 0.04 0.18

Intercept �0.38 �1.33 0.02 0.07 �0.04 �0.14

Observations 7216 7217 7216

KP gap equation

Daughter-in-law �0.04 �0.40 �0.07 �0.68 0.30 2.30��

H/h owns land 0.00 1.63� 0.00 �0.07 0.00 0.95

Farming household 0.14 3.44�� 0.07 �1.67� 0.12 1.99��

Durables index 0.00 �0.16 �0.01 �0.77 �0.10 �3.46��

No. of children 0.09 4.58�� 0.06 �2.69�� 0.02 0.53

Mother’s education �0.01 �0.30 0.03 1.01 0.04 1.38

Father’s education �0.04 �1.35 0.03 1.00 �0.04 �1.36

Elderly male in household 0.11 1.90� 0.03 0.60 0.03 0.35

Project area 0.13 1.25 0.03 0.26 �0.13 �0.58

Project� durables �0.01 �0.72 �0.01 �0.69 0.07 2.49��

Project� daughter-in-law �0.06 �0.57 �0.01 �0.09 �0.28 �1.81�

Project� children �0.03 �1.28 0.03 1.00 �0.01 �0.22

Intercept �0.95 �11.07�� 1.08 �10.17�� 1.57 5.66��

Selection term 1.44 5.82�� 1.89 3.33�� 0.61 2.74��


�Significant at greater than 10%, 5% and 1% levels, respectively.��Significant at greater than 10%, 5% and 1% levels, respectively.��Significant at greater than 10%, 5% and 1% levels, respectively.


DOI: 10.1002/jid

Figure 2. Mothers’ nutritional knowledge in project and control areas compare.


specific practice was modelled as a function of mother’s age, education, education of the

household head, whether the woman was a group member and participation in project

activities, including attending an antenatal clinic.24

The coefficients and the statistical tests of the variables ‘attending nutrition meetings’

and ‘supplementary feeding’ in Table 5 confirm that attending nutritional counselling has a

significant impact on a woman’s nutritional knowledge, though being in receipt of

supplementary feeding does not.25 However, even when these participation variables are

included, the BINP project dummy is still significant. This means either that there are

spill-over effects within the project area (women who get the knowledge in nutrition

sessions communicate it to others) or that other project activities not captured in the

participation variables, e.g. women’s group meetings, are also channels for communication

of nutrition education. According to these regression results, simply living in the project

area raises a women’s probability of having a piece of nutrition knowledge by 7 per cent,

but full participation in project activities increases this probability to between 10 and 23 per

cent (Figure 2). That is, the proportion of women aware of the importance of colostrums

feeding is 23 per cent greater for women participating in project activities than for women

in the control area. Other determinants of women’s nutritional knowledge are found to not

vary much between the project and control and so account for only a small amount of the

difference in knowledge between the two areas.

In summary, knowledge of the prescribed nutritional practices is higher in the project areas

than the control, the difference being greatest for women who have participated in project

activities. But this knowledge is not universal, which is consistent with the evidence of weak

links in the causal chain through incomplete enrolment, partial mis-targeting of supplementary

24One striking impact of BINP appears to have a much greater utilisation of antenatal care services, which isattributed to the one-stop service provided under BINP. In practice, this means that the CNP co-ordinated withhealth workers to provide services at the same time and place. Exposure to ANC under these circumstancesimplies exposure to nutritional messages.25That is, women could be in receipt of supplementary feeding without acquiring nutritional information. Thisfinding is not a good one for BINP, since supplementary feeding was intended mainly as a tool for demonstratingfeeding practices, not as a feeding programme. However, it was already shown that one-third of mothers whosechildren were enrolled in supplementary feeding did not discuss nutrition with the CNP.


DOI: 10.1002/jid

Figure 3. Knowledge-practice gap in project areas for selected nutritional practice.


feeding (and so the associated counselling) and failure to transmit counselling messages to all

women whose children have nutritional problems. A final weak link in any education-based

programme is of course failure by participants to absorb the intended messages.

5.2. Turning Knowledge into Practice: The Knowledge Practice Gap

Although the project has had success in promoting nutritional information, a considerable

knowledge practice (KP) gap remains: that is women do not put into practice things they

say they agree with (Figure 3). This gap exists for every practice, and is extremely large in

the case of exclusive breastfeeding. The gap exists in both project and control areas, with

little evidence that the gap is any less in project areas than control (Table 6).

The qualitative study found that harmful health beliefs, such as eating less and working

more during pregnancy, were on the decline, and were more prevalent among older women.

Various sources of information were identified as important for better knowledge,

including BINP but also the diffusion of mass media, national campaigns and health

workers. Regarding the knowledge-practice gap, women pointed to the lack of resources as

the reason for their inability to eat more during pregnancy. Women also mentioned

excessive workload and family opposition to the need for taking rest when pregnant. As we

now show, the quantitative data confirm this picture.

Multivariate analysis of the KP gap was carried out on a sub-sample of women saying

they have a piece of knowledge. These ‘knowledge-practice gap’ equations were estimated

for a sample of 7216 mothers in the Save the Children survey. The determinants of the gap

Table 6. Size of knowledge-practice gap in project and control areas compared

Project Control t-statistic

Rest during pregnancy 30.7 34.7 2.2��

Colostrum 18.3 21.1 1.3

Exclusive breast-feeding 95.7 94.2 1.1

Source: Calculated from Save the Children data. Note: �significant at 10%.


DOI: 10.1002/jid


were estimated using a probit regression, in which the dependent variable is one if there is a

gap (i.e. knowledge is not put into practice), and zero otherwise. The sample for this

estimation (‘KP gap equation’) is composed only of those mothers who have knowledge

regarding nutrition practices (between 60 and 75 per cent of the sample of 7216 mothers,

depending on the knowledge considered). The estimation of this model could produce

biased coefficients, because unobserved determinants of the knowledge could have effects

on the practice as well. The solution to this problem is the estimation of a

maximum-likelihood probit with sample selection. That is we first estimate the

determinants of the knowledge of the practice (the knowledge ‘selection equation’) for

the full sample of mothers, and the determinants of the gap including a selection correction

term, for the sample of mothers having the knowledge (see, e.g. Greene, 2000).

The results, shown in the bottom half of Table 5, identify a number of constraints on turning

knowledge into practice. Resource and time constraints are foremost among these. Women,

who have work to do, including children and elderly relatives to care for, are less likely to be

able to rest or avoid hard work during pregnancy. Women engaged in agricultural work may

also not have time to breastfeed, or not able to do so if they are with the child away from the

home. Daughters-in-law are less likely to breastfeed in non-project areas, but there is no

‘daughter-in-law effect’ in project areas; since nutritional counselling was seen not to reach

mothers-in-law, this result may show that access to outside knowledge may indeed empower

women in decision-making, which is a desirable outcome. Having more children increases the

probability of feeding the colostrums, which may indicate familiarity with the practice, or that

women that already have a child are more aware following birth.

The inter-sectoral nutrition component was partly intended to address the resource

constraint by increasing food availability. However, the coverage of the scheme was much

less than originally envisaged. And such activities were poorly targeted so that over half the

beneficiaries were not amongst the nutritional target groups. The assessment by the Bank’s

own operational staff responsible for the project concludes that that the level of inputs was

insufficient to bring about any improvement in the nutritional status of children in

beneficiary households (World Bank, 2002b).

In summary, as shown in Figure 2 and Table 5, the project does increase knowledge

about nutritional practices. However, there is a gap between knowledge and practice, and

the project does not have an impact in reducing the size of the gap. But since knowledge is

more widespread, and the gap the same in project and control areas, then the promoted

practices are more widespread in the project area. So, are these practices beneficial to

nutritional outcomes?

6. THE NUTRITIONAL IMPACT OF BINP INTERVENTIONS

6.1. Child Nutrition

Existing studies have found somewhat different results with respect to the impact of

BINP.26 Using an ex-post single difference analysis of project and control, Save the

Children found no evidence of impact on nutritional outcomes. A similar conclusion was

26The discussion here is restricted to anthropometric outcomes. The project also provided Vitamin A supplements,but data have not been collected on this outcome. Impact would have, however, been limited owing to the fact thatthese supplements were already distributed by a national project, which had achieved coverage in excess of80 per cent.


DOI: 10.1002/jid

Figure 4. Single difference estimates of project impact of BINP and IEG evaluations compare.


reached using the same methodology by the government’s own evaluation department,

IMED (Haider et al., 2004).27 However, the BINP evaluation finds a significant positive

impact from single difference estimates at both midterm and end line, and a significant

double difference impact on severe malnutrition.

The IEG study used the nationally representative NSP data to construct a control group

using propensity score matching methods based on the participation equation in Table 3.28

Figure 4 compares the IEG and BINP evaluation single difference estimates for both

average height for age (HAZ) and weigh for age (WAZ) for midterm and the end line. The

BINP evaluation estimates show a significant effect on weight for age at midterm, but not

height. But by the end line there is a significant effect on height for age. One would expect

the impact on weight to exceed that on height, but not expect the latter to be negative when

the former is positive. Hence neither of the BINP evaluation results make intuitive sense.

The BINP evaluation estimates are biased by the use of small control group, by the

survey being conducted at different times of the year, an by ignoring the selection bias

implicit in comparing project and control areas independently of participation. Our study

side-stepped these problems by using the BINP evaluation project data, but constructing a

new control. The IEG analysis finds positive impacts on both WAZ and HAZ at midterm

and end line, with the effect on WAZ being the larger of the two (Figure 4). These results

are more internally consistent than those from the BINP evaluation data, suggesting that the

method of constructing the control has produced more reliable results. But the impact

found is very small: the project is found to have reduced malnutrition by less than 2 per cent.

27Unlike the BINP evaluation and Save the Children data, the sample for this study was drawn from thanas inlater phases of the project. Neighbouring thanas were used for the control.28The exact specification differed to allow for the overlap of common variables between the two surveys. SeeWorld Bank (2005a): Annex G) for details.


DOI: 10.1002/jid

Table 7. Determinants of weight gain during supplementary feeding

Variable Coeffecient t-statistic

Female child 0.07 1.73�

Month at which supplementary feeding started

February 0.11 1.12

March �0.10 �1.11

April �0.06 �0.60

May �0.01 �0.18

June 0.20 2.12��

July 0.17 2.16��

August 0.09 1.12

September 0.10 1.04

October 0.05 0.69

November 0.17 1.73�

December 0.19 2.31��

Age at which supplementary feeding started

From 12 to 17 months 0.25 7.06��

From 18 to 23 months 0.28 4.76��

Nutritional status at start of supplementary feeding

Normal child (above �2 z-score) 0.24 1.05

Between �2 and �4 z-scores 0.57 2.51��

Below �4 z-score 0.91 3.70��

Child shared the food with other household members 0.00 �0.11

Child was not given other food than supplementary food at meal time �0.07 �1.91�

Child was ill during supplementary feeding �0.02 �0.46

Child missed some of the sessions 0.00 �0.74

Constant �0.52 �2.27��

R-squared 0.21

Observations 716



This is far less than planned, and indicates a low level of cost effectiveness. Simulations suggest

that simply buying food and giving it away to families with children would have reduced

malnutrition by the same amount for half the cost, even allowing for administration costs and

higher leakage of such a scheme (World Bank, 2005a).

It proved difficult to construct a control for the supplementary feeding programme. Data

on growth faltering among the control were not available, and the determinants for the

selection equation did not to work well, so the problem of endogeneity of programme

placement remained, resulting in an apparent negative programme impact. The best

source for analysis is the Save the Children register data, which includes the weight for age

z-scores of all children who had received supplementary feeding, both during the feeding

and at other times. The regression results for the change in the WAZ between starting and

ending feeding are shown in Table 7.29,30

29Time of permanence in supplementary feeding varied considerably, though 87 per cent of the sample childrenwere fed for between 3 and 6 months. A model in which the dependent variable was divided by the number ofmonths of permanence in the programme produced results analogous to those shown in Table 7.30The Save the Children sample of children who received supplementary feeding consisted of 998 children.However, after dropping censored observations (children who, because of their age, were not fully exposed to foodsupplementation), observations for which supplementation history was not complete, and cases with missingobservations, the final sample reduced to 716 children.


DOI: 10.1002/jid

Table 8. Weight gain from supplementary feeding

Nutritional status Child’s age

Malnourished �0.11 <12 months 0.02

Severely 0.22 12–17 months 0.27

Very severely 0.56 17–23 months 0.29

Source: Calculated from Table 7.


The main result is that supplementary feeding is much more effective for children who

are malnourished and particularly those who are severely malnourished. The results also

provide evidence of the adverse effects of substitution. Finally, there are very striking

seasonal effects from the weight gain experienced during supplementary feeding, with a

larger impact in the lean seasons.

The regression results can be used to calculate the weight gain during supplementary

feeding for a child with average characteristics. Table 8, shows the results, varying only the

characteristic shown. Children who enter the programme more malnourished gain more from

the programme than better nourished children.31 Children under 12 months appear to show

negligible gain in WAZ during the programme, compared to close to 0.3 WAZ for older

children. However, the NSP data show that the WAZ falls for children across Bangladesh at a

rate of about 0.5 WAZ over a 3-month period.32 This occurs since the curves in the

growth-monitoring chart are very steep for children aged below 12 months, so that substantial

weight gain is needed to not fall further below the curve. Hence, compared to this reference

group, a zero gain of WAZ is actually a gain of 0.5 WAZ, which is not insubstantial.

6.2. Low Birth Weight

BINP aimed to increase pregnancy weight gain by both encouraging women to eat more

during pregnancy and providing supplementary feeding to malnourished pregnant women.

Tables 9 and 10 present multivariate analysis conducted using pooled BINP data of the

midterm and end line surveys. The analysis of the pooled data includes a project dummy

which represent the project effect (called ‘BINP area’), a time dummy (called ‘time’),

which represent the effect of unobservable determinants over time, and the interaction

between time and project dummies, which measure the differential change in the outcomes

over time in project areas with respect to control areas; that is the double difference (also

called ‘difference in difference’).

Table 9 shows an increase of pregnancy weight gain of about 200 grams per month in BINP

areas,33 mainly through the fact that women in project areas were, as shown above, more likely

to eat more during pregnancy than those in the control.34 However, the project-related weight

gain is not much more than 1 kilogram over 6 months, which, evidence suggests, increases

birth weight by only 20 grams. That is, the amount of pregnancy weight gain appears

31The category of normally nourished is not shown in Table 8 as they constitute less than 1 per centof programme participants.32That is, children in Bangladesh do not, on average, gain weight at the rate that the growth charts (based on USdata) suggest they should. The constant in the Table 7 regression is �0.5, showing a remarkable consistencybetween the two datasets.33This is the combined effect of the variables ‘BINP area’, and ‘Mother ate more during pregnancy in BINP areas’in Table 9.34Participation in supplementary feeding was not found to have a significant impact. This impact may have beenmuted by leakage and substitution.


DOI: 10.1002/jid

Table 9. Determinants of pregnancy weight gain

Midterm Endline Pooled data


Mother’s height 4.5 1.99�

Mother’s age �4.5 �1.34 �6.6 �2.12�� 4.8 �1.70�

4th gestational month 110.7 1.38 433.9 7.90�� 129.5 2.01��

5th month 548.8 7.15�� 243.7 4.50�� 455.4 7.38��

6th month 709.1 9.35�� 332.6 6.08�� 607.6 9.94��

7th month 992.9 13.09�� 293.6 5.33�� 849.0 13.88��

8th month 1027.0 12.79�� 477.4 8.07�� 899.6 13.83��

9th month 1179.6 10.80�� 713.2 4.47�� 1081.5 11.37��

3rd and 4th week of January 226.0 2.59�� 219.6 2.67��

1st and 2nd of February 454.8 5.22�� 463.8 5.51��

3rd and 4th of February 305.5 6.01�� 315.0 6.37��

1st and 2nd of March 103.1 2.18�� 107.2 2.32��

3rd and 4th of March �85.7 �1.27 �26.4 �0.30

1st and 2nd of April �55.3 �0.87 30.8 0.28

Middle income 75.6 1.89� 85.3 2.14�� 68.2 2.00��

High income 127.8 1.95� 73.0 1.01 115.9 2.04��

Landless �97.7 �2.63�� 1.6 �0.04 �81.0 �2.53��

BINP area 162.0 2.84�� 136.8 3.00�� 156.8 2.92��

Mother ate more during

pregnancy in BINP area

73.8 1.77� 67.4 1.70� 76.1 2.12��

Time 332.0 2.84��

Difference in difference �74.5 �0.72

Constant 298.1 2.37�� 273.2 0.78 392.4 3.59��

R-square 0.06 0.10 0.06




insufficient to have a substantial impact on low birth weight. In addition, multivariate analysis

(Table 10) shows there are important factors, such as gestational period and month of birth,

which affect low birth weight and which are outside the control of the project. Moreover,

international evidence suggests that pre-pregnancy weight matters more than pregnancy

weight gain for birth weight (e.g. Kramer, 1987).

Direct estimates of project impact on birth rate weight suggest a slight project impact,

mainly as a result of the mother not eating down. This impact was found to be much larger for

children born to very poor women (results not shown), as measured by a household assets

index. Women categorised as poor or destitute, have presumably a poorer nutritional status,

which is reassuring as women of low body-mass index (BMI) need a larger pregnancy weight

gain to avoid low birth weight than do women with a higher than average BMI. The project

also seems to smooth out the seasonal effects that come from mothers being less well

nourished in the lean season, which is most likely a result of supplementary feeding.

7. CONCLUSION

7.1. Explaining Low Impact

A number of reasons may be advanced for the apparent low impact of BINP: (1) spill-over

effects from project areas to control undermine measured impact, (2) weaknesses in


DOI: 10.1002/jid

Table 10. Determinants of birth weight

Midterm Endline Pooled data


9th gestational month 104.4 3.01�� 167.9 3.66�� 133.1 4.77��

10th gestational month 93.5 2.29�� 239.9 4.47�� 154.8 4.72��

Female child �58.0 �3.18�� 69.2 �3.60�� 61.2 �4.52��

Parity �34.2 �1.32 �108.5 �3.63�� 52.1 �2.66��

Age 13.4 1.19 14.3 0.91 20.5 2.11��

Age square �0.2 �0.89 �0.2 �0.83 �0.3 �1.88�

Mother ill 19.8 0.63

Pregnancy history �188.6 �2.26��

Father’s education 14.0 4.33��

Mother’s education �1.2 �0.33

Land owned 0.2 2.16�� 0.1 3.14��

Middle income 56.8 2.52��

Rich 115.7 2.79��

December 360.6 4.38�� 112.0 2.75��

January 355.7 4.42�� 81.2 0.63 111.0 2.84��

February 422.5 5.25�� 78.5 2.08�� 161.3 4.71��

March 283.5 2.94�� 67.8 1.91� 128.2 3.86��

BINP area �70.7 �2.72�� 36.6 �1.32 �52.2 �2.14��

More food during pregnancy 43.2 2.21�� 88.1 3.75�� 86.4 6.11��

Less food during pregnancy �44.5 �1.74�

Time 105.8 3.46��

Difference in difference �12.8 �0.40

Constant 2118.6 10.86�� 2387.9 10.51�� 2220 15.02��

R-squared 0.05 0.12 0.07

observations 1560 1362 2922

Notes: Birth weight is measured in grams. �significant at 10%, ��significant at 5%, ��significant at 1%.


implementation and (3) inappropriate design meaning that the programme would have had

at best a small impact.

Defenders of BINP (e.g. Levinson and Rohde, 2005; Sack et al., 2005) argue that word

on BINP’s success has spread so that similar programmes are being carried out in

neighbouring thanas (the Save the Children study used thanas adjacent to project thanas

for the control), or even that improved practices simply travel by word of mouth. However,

these arguments can be discounted for three reasons. First, both Save the Children and

IMED teams verified that similar projects were not taking place in the control areas.

Second, such spillover effects would have to be national to be picked up by our study,

which used nationwide data to construct the control. Finally, as shown above, the project

has resulted in differences in knowledge, which would not be the case if there were

spillover, since it is precisely through improved knowledge that spillover would operate.

Hence other reasons have to be sought for low impact, which are to be found in both weak

and missing links in the causal chain.

The longer the causal chain, the more likely it is that final outcomes will not be realised

on account of missing or weak links in the chain (Table 11), and since there are more

opportunities for external factors to undermine the ‘logical flow from inputs to outcomes’.

There were two missing links in the BINP chain: the first was the relative neglect of some


DOI: 10.1002/jid

Table 11. Links in the causal chain

Assumption Children Mothers

Attend growth monitoring

sessions

Over 90% of children attend

GM sessions

Over 70% participate in

monitoring pregnancy weight gain

Targeting criteria correctly

applied; participants stay in

programme to receive food

Nearly two-thirds of eligible

children not fed (reasons: don’t

attend GM in first place, wrong

application of targeting criteria,

drop out of feeding)

60% of eligible women not

receiving supplementary feeding

Acquire knowledge and put

it into practice

One-third of mothers of children

receiving supplementary feeding

do not receive nutritional

counselling. There is a

knowledge practice gap

(see mothers).

There is a knowledge practice gap,

driven by material resource or

time constraints.

No leakage or substitution One quarter of children fed

at home, increasing possibility

of both leakage and substitution.

One-third admit sharing food, and

there is substitution for those who

do not. At most 40 percent of eligible

women receive full supplementation.

Feeding and nutritional

advice have an impact on

nutritional status

Supplementary feeding has a

positive impact on child nutritional

status, especially for the most

malnourished children. There is only

weak evidence of any impact from

nutritional counselling.

Pregnancy weight gain is too little to

have a notable impact on low birth

weight, except for most malnourished

mothers. Moreover, mother’s

pre-pregnancy nutritional status is

more important than pregnancy

weight gain. Consequently,

birth weight gains are slight, though

greater for children of severely

malnourished mothers. Eating more

during pregnancy is the main channel

for both pregnancy weight gain and

higher birth weight.


key decision makers regarding nutritional choices (men and mothers-in-law), and the

second the focus on pregnancy weight gain rather than pre-pregnancy nutritional status.

Participation levels of the target audience were high, but many women escaped exposure to

nutritional messages, and there was a high Type I targeting error in the feeding

programmes. A substantial knowledge practice gap persisted, so many women did not put

the advice they received into practice, especially if they were resource or time constrained.

Those receiving supplementary feeding, often shared it with others or substituted it for

their regular foodstuffs. This list of weak links in the chain explain why project impact was

muted by the time final outcomes are considered. While attention can be paid to each of

these weak links, the BINP experience does demonstrate the difficulty of implementing

complex designs.

Both the IEG results and the BINP evaluation find a stronger impact on child nutrition at

midterm than end line. Between midterm and end line, pregnancy weight gain was greater

in control thanas than those falling under BINP. The impact of BINP appears to have waned

over time. One reason for this may be the difficulty of maintaining good implementation

over time, or as the programme is rolled out. Unfortunately the evaluation data set has few


DOI: 10.1002/jid

Figure 5. Rice production and daily energy supply in Bangladesh (1970–2002).


process variables to explore this hypothesis. A second explanation is that there were

external factors, which allowed non-project areas to catch up.

There is a clear candidate for such an external factor: daily energy supply grew rapidly in

Bangladesh in the second part of the 1990s as a result of a yield-driven increase in rice

production (Figure 5). Famine theory shows the importance of entitlements: it is not simple

food availability that matters, but whether people can afford to buy that food. In

Bangladesh real incomes have been growing throughout the 1990s, at over 5 per cent a

year, picking up in the second half of the 1990s. Although growth in Bangladesh has not

been pro-poor, real income growth has been above 2 per cent for the lowest income groups

(World Bank, 2002a). Meanwhile the rice price has been stable in nominal terms,

representing a real fall in the average rice price of 14 per cent from the early to late nineties,

and a further 15 per cent from 1999 to 2002.

Greater food availability, combined with rising incomes and lower prices, presents a

very plausible explanation for improved nutritional status across the country. It is possible

that at the time of the midterm these factors had not fully passed through to improved

nutritional status, allowing some effect of BINP to emerge. But by the end line, project

impacts had been dampened by improved energy intake in all areas since the project is most

effective for severely malnourished groups.

7.2. What is to be Done?

The limited impact of BINP and its cost raises serious doubts as to the justification for

scaling the project up to the national level in its current form. This limited impact points to

either or both failing in design and implementation, and hence shortcomings in Bank

performance. In order for the project to be justified then its efficacy and efficiency must be


DOI: 10.1002/jid


improved. In seeking to do this several lessons emerge from this analysis, which should be

borne in mind:35

1. S

35Arevfoll

Cop

upplementary feeding for children does have a positive impact, especially for the most

malnourished children. NNP has revised the eligibility criteria so that only growth

faltering children with WAZ<�2SDs receive feeding, as well as those with

WAZ<�4SDs. The latter group constitutes a very small percentage of children, so

there appears scope for raising this threshold to, say, �3SDs, to capture more children

among whom feeding seems most successful. There is also some mistargeting which is

likely to be best addressed by further training for Community Nutrition Promoters in

interpreting growth charts.

2. S
upplementary feeding for pregnant women appears a flawed approach on two grounds:
the pregnancy weight gain achieved is mostly too small to have a notable impact on

birth weight, and it is anyhow pre-pregnancy weight which evidence suggests to be the

more important determinant of birth weight. The programme would appear more

successful if it restricted its attention to the most malnourished of women, improved

targeting to reduce Type II targeting error and tried harder to discourage leakage and

substitution.

3. F
or both types of feeding programme, there is evidence of a greater impact in the lean
season. There are grounds for considering either increasing the size of the food

supplement in this period, restricting it to those months, or adjusting the eligibility

criteria by time of year.

4. D
iscouraging women from eating down during pregnancy has some benefit for birth
weight. But all forms of knowledge transmitted by the project suffer from a knowl-

edge–practice gap, so attention need to be paid to both the resource constraints which

create this gap and transmitting knowledge to other key actors: mothers-in-law and

husbands.

5. H
owever, based on current performance, scaling up will prove very costly, with limited
nutritional gain. If efficiency cannot be improved through better implementation then

there are other ways of spending the money, which would have a greater nutritional

impact, and other channels could be sought for conveying nutritional information,

tailoring it to the differing needs of different socio-economic groups.

7.3. Lessons for Evaluation Practice

This study illustrates the benefits of a theory-based impact evaluation, i.e. one which traces

the links from inputs through to impacts, rather than only look for evidence of impact. So

doing allows identification for the reasons why the programme is working or not, and so

gives indication of needed corrected actions. The implication is that evaluation design

needs begin with an elaboration of programme theory and data collection cover the whole

causal chain. Where an intervention has its own logical framework, this framework may

serve as a basis for the programme theory, though some aspects may need further

elaboration or indicators identified.

t the time of writing (June 2005) World Bank operational staff responsible for the project are engaged in aiew of all the evaluative evidence regarding the project to make decisions regarding implementation of theow-on National Nutrition Project.


DOI: 10.1002/jid


The controversies surrounding the various BINP evaluations show both the importance

of a good control group and the difficulty of obtaining one. An increasingly popular

solution to this problem is to randomise the project design: project beneficiaries are a

random sample of a larger population, so that selectivity bias does not arise. This approach

certainly has its advantages, though also some drawbacks (see, for example the discussion

in the various chapters in Pitman et al., 2005). Hence whilst randomisation should be

considered where feasible, there will remain many cases in which other approaches have to

be used to establish a satisfactory control group. Our study has shown the improvement

made in the control by using propensity score matching.

REFERENCES

Berg A. 1987. Malnutrition: What Can Be Done? Lessons from World Bank Experience. The John

Hopkins University Press: Baltimore.

Carvalho S, White H. 2004. Theory-based evaluation: the case of social funds. American Journal of

Evaluation. 25(2): 141–60.

Chopra M, Sanders D. 1997. Growth monitoring—is it a task worth doing in South Africa? South

African Medical Journal 87: 875–878.

Greene WH. 2000. Econometric analysis (4th edn). Prentice-Hall: Upper Saddle River, NJ.

Haider SJ, Hussain D, Nayer I, Maleque A, Ashaduzzaman Md, Khan AR. 2004. Impact Evaluation

of Bangladesh Integrated Nutrition Project, IMED and Read, Dhaka.

Hossain SM, Duffield A, Taylor A. 2005. An evaluation of the impact of a US$60 million nutrition

programme in Bangladesh. Health Policy and Planning 20: 35–40.

Karim R, Lamstein SA, Akhtaruzzaman M, Rahman KM, Alam N. 2003. The Bangladesh Integrated

Nutrition Project: Endline Evaluation of the Community Based Nutrition Component, The

Institute of Nutrition and Food Sciences University of Dhaka, Bangladesh and The Friedman

School of Nutrition Science and Policy, Tufts University, USA.

Kramer MS. 1987. Determinants of low birth weight: methodological assessment and meta-analysis.

Bulletin of the World Health Organisation 65(5): 663–737.

Levinson FJ, Rohde JE. 2005. Letter to the Editor. Health Policy and Planning 20: 405–406.

Martorell R, Shekar M. 1992. Growth-faltering rates in California, Guatemala, and Tamil Nadu:

implications for growth-monitoring programs. Food and Nutrition Bulletin 5(3): http://www.unu

edu/unupress/food/8f153e/8F153E02.htm#Growth-faltering%20rates%20in%20California,

%20Guatemala,%20and%20Tamil%20Nadu:%20Implications%20for%20growth-monitoring%

20programmes

Mascie-Taylor Nicholas. 2004. July 2004 Dhaka presentation.

Panpanich R, Garner P. 1999. Growth monitoring in children. The Cochrane library 2003 (1). Update

Software: Oxford.

Pearson R. 1995. Thematic Evaluation of UNICEF Support to Growth Monitoring, UNICEF,

Working Paper Series, No. 2.

Pitman GK, Feinstein O, Ingram G (eds). 2005. Evaluating development effectiveness. Transaction

Publishers: New Brunswick, N.J.

Rosembaum PR, Rubin DB. 1983. The central role of the propensity score in observational studies for

causal effects. Biometrika 70(1): 41–55.

Sack DA, Roy SK, Ahmed T, Fuchs G. 2005. Letter to the Editor. Health Policy and Planning 20:

406–407.


DOI: 10.1002/jid


Save the Children. 2003. Thin on the Ground. Questioning the Evidence Behind World Bank-funded

Community Nutrition Projects in Bangladesh, Ethiopia and Uganda, Save the Children UK,

London.

Weiss Carol. 1998. Evaluation: methods for studying programs and policies. Prentice Hall: New

York.

World Bank. 1994. Tamil Nadu Integrated Nutrition Project: Impact Evaluation Report. Report No.

13783-IN. Washington, D.C.: OED, World Bank.

World Bank. 1995. Staff Appraisal Report. Bangladesh Integrated Nutrition Project, Population and

Human Resources Division, Washington D.C.

World Bank. 2002a. Poverty in Bangladesh: Building on Progress. World Bank. Washington D.C.

World Bank. 2002b. Bangladesh Integrated Nutrition Project: Implementation Completion Report.

Report No. 25183, Washington D.C.: World Bank.

World Bank. 2005a. Maintaining momentum to 2015? An impact evaluation of interventions to

improve maternal and child health and nutrition outcomes in Bangladesh. Operations Evaluation

Department, World Bank. Washington D.C.

World Bank. 2005b. Bangladesh Integrated Nutrition Project: Project Performance Assessment

Report, Report No. 32563, Washington D.C.: Operations Evaluation Department, World Bank.


DOI: 10.1002/jid

assessing interventions to improve child nutrition: a theory-based impact evaluation of the...

Documents