Slide 1

Silvia Alayn20th International Congress on NutritionGranada, Spain September 18, 2013

Alternatives to RCTs: Evaluation of A&Ts national mass media campaign, lessons from Viet NamCurrent debate about real world applicability of RCTs, especially in the context of programs that are truly operating at scale. So I hope that this presentation will get you thinking about how we can be systematic and rigorous in our evaluations, contributing to the evidence base for what works, even when that gold-standard of the RCT cannot be met.1Presentation SummaryViet Nam contextMass media campaignEvaluation challengesEvaluation approachesPreliminary resultsLessons learned

2

Viet Nam BackgroundTotal Population (2011): 88 million in 63 provinces31% Urban (2011)Literacy rate: 93% (2011) Total Fertility Rate (2011): 2 children/womanInfant mortality rate (2011): 17/1,000 live birthsLife expectancy (2011): 75 yearsTelevision ownership (2005): 85%Weekly television viewership (2011): 92%Stunting (2010): 29%Exclusive Breastfeeding (2010): 20%

3

The population in Viet Nam is a middle income country that is literate, modern, rapidly urbanizing. Viet Nam has achieved declines in fertility, high access to health services (e.g. immunization=95%) and increased life expectancy, but stunting among young children remains high at 29% in part due to poor IYCF practices. In particular, EBF rates are far from optimal. Only 1 in 5 infants under 6 months is exclusively breastfed.

In this context, A&T decided to emphasize exclusive breastfeeding, using a mass media campaign to achieve scale.

Source: Viet Nam GSO web site, accessed on 9/5/2013; 2011 MICS; 2010 NIN National Nutrition Surveillance; 2005 AIDS Indicator Survey3A&T Objectives in VietnamIncrease exclusive breastfeeding to 27.5%, from 19% in A&T areas at baseline Prevent stunting in 54,000 childrenReach 2.5 million families of children under 2: 1 million through interpersonal communication delivered at 781 IYCF franchises in 15 provinces1.5 million through a national mass media

4Achieving these objectives required operating at scale (there are only a little more than 2.5 million children under 2 in Viet Nam at a given point in time). Though A&T established 781 IYCF franchises and an additional 600 community-based support groups to deliver interpersonal communications, these cover less than 10% of children under 2.

With such high access to mass media, especially television, it was an ideal setting to learn the extent to which mass media could have an impact on exclusive breastfeeding. 4Alive & Thrive Model5

Evaluation of Mass MediaThis is the Alive & Thrive Program model, which Purnima showed earlier. She presented some results related to the FLWs, which deliver the interpersonal communications component. Now I will focus on the mass media component. 5A&Ts behavior change model

This model is adapted from the theory of reasoned action. It shows that in order for the mass media campaign to result in behavior change, we not only had to ensure that women had correct knowledge about exclusive breastfeeding, but also change their beliefs about the benefits of EBF, what they perceived the social norms to be and their own self-efficacy. These are what we refer to as behavioral determinants.

Now Ill move on to the evaluation challenges and our approaches.

We also view EBF as a group of component behaviorsnot giving water, not giving formula, etc. During the formative research for the campaign, water was identified as the biggest threat to EBF, due in part to womens entrenched beliefs about BF babies being hot, thirsty or getting thrush if not given water. This was confirmed when we looked very carefully at the baseline data.

6Elements of the mass media campaignNumber of TV Commercials (4)2 promoting exclusive breastfeeding for 6 months 1 promoting consumption of iron rich foods and 1 promoting the IYCF franchisesMedia channels usedNational & provincial televisionVillage loud speakers (rural)Out of home advertisement (billboards, bus wraps, etc.)Video screens in supermarkets (urban)Internet (urban)Key messagesBreastfed infants (under 6m) do not need waterNursing more will ensure sufficient breast milkFeed infants and young children iron rich foodsOther messagesTarget perceptions about: benefits of breastfeedingsocial normsself-efficacy 7

The MM campaign was launched in August 2011 with the MTBT website. This was followed by TV spots in November 2011.7Evaluation challengesInability to randomize, universal coverage of national TVMany interventions occurring concurrentlypolicy changes*, national program, etc.

8

*Poster on Building political commitment for longer ML and marketing of BMS5-6pm, Wednesday, Screen 6.The fact that regular TV viewership was so high, made Viet Nam the perfect context within which to launch a mass media campaign. This also created challenges in the evaluation.

Furthermore, the complex nature of the program, which had other components also operating at scale, makes it difficult to attribute, with certainty, changes in behaviors or in behavioral determinants to the media campaign.

I want to add that the evidence for the use of mass media to influence breast feeding was very limited at the time, so A&T and the Gates Foundation felt that it was important to rigorously evaluate the campaign to add to the thin evidence base.

On Building political commitment and consensus for longer maternity leave and the marketing of BMS. Its from 5-6 second floor. Screen 68Four Evaluation ApproachesAnalysis of national EBF trendConstructed cohort designGeographic cohort designTrends & associations along the causal pathway: exposure to the campaign, behavioral determinants and EBF

9

To overcome those challenges we consulted with experts in behavioral sciences and communications. They recommended four approaches. On its own, each approach has weaknesses and none allows us to definitively attribute results to the campaign. But if we observe convergence in results across the four approaches it makes a stronger argument for the plausibility that the mass media had an effect on EBF.

This evaluation is ongoing and will not be complete until next year so I wont be able to tell you how the story ends, but in the sections that follow, Ill describe our experiences with each the approaches, present the data available to date and share lessons learned so far.9Data Sources10Interim 14 provincesInterim 2 4 provincesInterim 34 provincesMass media Campaign1These are two main sources of data for this evaluation. One is the data from Viet Nams National Nutrition Surveillance Program. Since 2010 the Viet Nam National Institute of Nutrition has collected EBF data according to standard WHO definitions. NIN conducts annual Nutrition Surveillance Surveys, collect national level information on a limited number of indicators--WHO core indicators and anthropometrics. So, trend data on EBF are available from prior to the campaign. In our surveys, we put a lot of emphasis on: a) fine tuning the measurement of the behavioral determinants and b) identifying which were the most important behavioral determinants to track over time so that wed have data that covered the entire pathway described in the behavioral model.10Approach #1: Analysis of National Trend (%EBF)11This is an example of Approach 1, the trend analysis. One strength of this approach is that it uses available data, represents the national picture. Another strength of this approach is that, assuming NIN will continue the surveillance program, anyone can extend the evaluation to understand the long-term impact of the campaign. The weakness is that it does not provide evidence linking the trend to the campaign, making it difficult to disentangle the effect of the media campaign from those of other national-scale interventions such as policy changes. Furthermore, though we expected the data from NIN to be available, it has been difficult to obtain. So two years after the launch of the campaign, we have yet to obtain access to pre-campaign data. Post-campaign data have yet to be published by NIN. To overcome this barrier, here we present trend analysis using a combination of nationally representative sources.

Here, the blue line represents nationally representative data from DHS (1997 & 2002) and NIN (2010). The slope of this line, the historical trend in EBF is flat (0.014-0.022 percentage points per year, depending on whether we calculate it through 2010 or 2011. Either way, its pretty flat.

The green dotted line is dummy data showing what wed expect to see after the campaign. I wont spend too much time on this slide, since it is dummy data. Luckily, A&T had invested in other approaches as well.

11Approach #2: Constructed cohort designUsing post-campaign data (e.g. TV viewership habits, etc.) to predict the likelihood of exposure among pre-campaign respondents Based on prediction, create two cohorts from the baseline data: likely exposed and likely unexposedDisaggregate baseline data by these two groups to simulate comparable groups from the baseline data12Strengths of this approach are that. Our data are cross-sectional. This approach would simulate longitudinal data and allow us you to simulate what the exposed group would have looked like (behaviors, beliefs, etc.) prior to the campaign.

Predicting exposure to the campaign, even with two rounds of post-campaign data was difficult. Due in part to the high exposure and very little variability in outcome. 80% - 85% of mothers were exposed, with little variability based on geography, age, etc. Our data are very poor at predicting likelihood of exposure.

After exploring several alternative exposure measures, this approach will likely be dropped and replaced by something else (perhaps matching).

12Approach #3: Geographic cohort designUse natural variability in exposure among geographic areas to create areas of high and low exposureCompare trends between the high and low exposure geographic areas

13

This is very similar to approach three, but groups will be based on districts, rather than individuals. The results are highly confounded by whether the area is an A&T district with a franchise and other A&T inputs or not. About 12-14% of women in the total sample have visited a franchise, but almost all of them had been exposed to the campaign. Even with this low percentage, exposure to the campaign is significantly higher in A&T districts (84% vs. 75% exposure to TV) and though there are differences among these high/low areas, we cant distinguish at the moment between the effect of the campaign and the effect of other district level interventions. To estimate the effect of the mass media campaign (without the other) we will likely remove those women who had visited a franchise from the analysis. This will remove some of the noise, but cannot account for all of it, since other things like community events to promote BF are also more likely to occur in A&T districts.

13And now for the exciting results. . .14Approach #3: Geographic cohort designExposure to EBF campaign by district (2012)15n=2,065Highlight HIGH Exposure. 3 districts above 90% exposure. Only 1 district below 65%.

Green=A&T districts; Blue=non-A&T districtsn=2065Source: R1 interim data (4 provinces only)

15Aggregate cohort designEBF by levels of exposure (2012)16n=xxxx; p