Evidence on ImpactScaled mHealth Initiatives

Dr Eugene Sickle(on behalf of the RHI mhealth team)

Wits Reproductive Health & HIV Institute11 December 2014

“mHealth represents the collision of two interesting worlds — mobile, which changes on what seems to be a daily basis, and healthcare, which changes infrequently, only after significant deliberation and usually much empirical analysis”Joseph Kvedar, Center for Connected Health

“MAMA brings together leaders from across corporate, nonprofit and government sectors; harnessing their

strengths and assets to empower new and expectant mothers with access to vital health information”

http://mobilemamaalliance.org/

Bangladesh, India and South Africa

High mobile phone penetrationWill not meet MDG4 and 5 targets

In South Africa

Four Channels of delivery

SMS – twice weekly messages, pushed to user phone, until infant turns 1 Recruitment at clinicsInformed consentResearch plan & ethics approval

USSD - interactive quizzes

Mobi – web-based communityStand-alone and hosted on Vodafone Live (free to Vodacom users)

Mxit- interactive, stage-based platform

ChannelRegular

messagesHIV

messagesPre-natal Postnatal Total

SMS 8802 (60.6%) 5722 (39.4%) 8279 (57%)6245 (43%)

14,524

Registered Not registered Pre-natal (of registered)

Post-natal (of

registered)

Webpage 31,815 (7%)473,718 (93%)

4,808 (15.1%)

27,007 (84.9%)

411,813

Mxit 4268 (4.2%)96,423 (95.8%)

1208 (28.3%)

3060 (71.7%)

100,619

USSD11,236 (100%)

11,236

Total 538,192

Message content is timed to the expected month of delivery

vaccination check-up reminders exclusive breast feeding recommendations psycho-social parenting tips baby development and more

72 pregnancy related messages102 post-delivery messages

“If you are asking a lot of questions [via SMS], people stop responding. We found that four questions worked well. We saw that with every extra question beyond that the respondents dropped off.” - Bhupendra Sheoran, Youth + Tech + Health

Measuring Impact

“Look at the indicators the health service is using, and use those same indicators.” – Isaac Holeman, Medic Mobile

Purpose:Measure the impact of MAMA bi-weekly stage-based SMS's on MAMA SMS recipients compared to non-recipients (receiving standard of care)

Main study outcome is the degree of completion all four immunizations completed at 12 months post-natal

Total number of ANC visits (>4 visits) Individual infant immunization attendance (at 6,

10 and 14 weeks, as well as 9 months) Infant anthropometric measures Maternal Haemoglobin values during ANC, TB

status & breastfeeding questionnaire WHO Quality of Life measures (WHO-QOL-BREF

tool)

Secondary Outcomes

Some Preliminary Data - Gestational Age at Birth

Some Preliminary Data – Weight at Week 10 (kgs)

The study is powered to show a 10% increase (from 80% to 88%) for all 4 immunisations at 12 months

Full results expected April 2015

?

Acknowledgements

Partners

#HealthForAll

BBC Media Action

BBC’s international development organization

Aims to transform lives through media by giving people a voice and access to information

Reaches 200 million people

Active in 30 countries, 25 languages in Africa, Asia, Europe and Caucasus, Middle East and North Africa

All media platforms, linked with community activation and interpersonal communication

Health

Resilience and humanitarian responseGovernance and Rights

Yvonne MacPhersonDirector, USA@YMacPherson

Shaping demand and practices in Bihar

The 'Ananya' programme – a collaboration between the Bill & Melinda Gates Foundation and the Government of Bihar – aims to significantly reduce maternal and infant mortality in the state of Bihar by Dec 2015.

Context: Bihar has a population of 104 million people, including 27 million women of child bearing age makes the scale of this challenge immense.

As an Ananya partner, BBC Media Action is working to:

• Create demand for health services• Increase the adoption of healthy

behaviors

Mass Media

Community Mobilizatio

n

Interpersonal Communicati

on

Pregnant women, mothers

(children <2 years)

Mob

ile

Frontline Health Workers – key to behavior change

• 200,000 frontline health workers in Bihar

• Responsible for home visits to the most remote communities in need of health information

• They have the enormous task of:– delivering critical health

information to families – and promoting the adoption of

life-saving behaviours

• But they have little support to do this– Lack training and job aids– Lack respect in the community

Mobile Kunji is an audio-visual job aid designed to enhance the impact of frontline health workers during their interactions with families

• Mobile Kunji enables health workers to deliver high quality standardized health counselling

• Mobile Kunji brings together an IVR-based mobile service and a virtually indestructible deck of 40+ illustrated cards on a ring

• Each card communicates life-saving messages about maternal and child health

• A unique code at the bottom of each card plays the related audio health message to rural families

• Mobile Kunji can be used from any mobile phone

• Call charges are paid for by the Government of Bihar

Mobile Kunji

Take up and usage of Mobile Kunji in Bihar

• 55,000 frontline health workers trained in Bihar

• 182,000 unique users

• Almost 40,000 regular monthly users

• Playing more than 700,000 minutes of content per month

• More than 11.4 million minutes of content played to date

Mobile Kunji uptake by health workers

Almost 40% exposure to Mobile Kunji in most recent home visit

• The survey found that among all women who had received a home visit from an health worker in the last six months, almost 40% had been exposed to Mobile Kunji.

Home visits in which Mobile Kunji is used tend to be longer, more comprehensive

• On an average households exposed to Mobile Kunji received longer home visits from frontline health workers

15 10 10minute average duration of a home visit where Mobile Kunji has been used

minute average duration of a home visit in focus districts where Mobile Kunji has not been used

minute average duration of a home visit in non focus districts

Use of other Ananya tools by health workers (such as the complementary feeding bowls, uterus models, IUDs) is 7-10 times more likely if the health worker is also using Mobile Kunji

Mobile Kunji exposure strongly correlated with delivery preparation

• The survey found that when pregnant women in focus districts were exposed to Mobile Kunji in the last six months, there was a 28% increase in the number of pregnant women who prepared for birth compared to baseline

57.8

30.0

MK used in past 6 months Mk not used in past 6 months

Prepared for Delivery Adjusted

Percentage

Delivery preparation: save money, arrange transport, save important phone numbersBase = mothers of children aged 0-6 months

Mobile Kunji exposure strongly correlated with complementary feeding

• The survey found that when mothers of infants were exposed to Mobile Kunji in the last six months, there was a 13.5% increase in the practice of complementary feeding

80.4

66.9

MK used in past 6 months

Complemen-tary feeding

practiced Ad-justed Per-

centage

Base = mothers of children aged 0-6 months

• The use of the Kunji was not found to be significant for two behaviors: – facility delivery;– immunization (specifically administration of DPT3 vaccine).

The adoption of these behaviors is intrinsically linked to supply side management.

Impact of Mobile Kunji not significant for facility delivery and vaccination

It’s worth noting that:

• There was no difference between current reported levels of facility delivery and administration of the DPT3 vaccine between focus and non-focus districts for the Ananya Program as a whole (irrespective of Kunji exposure).

• Facility delivery is linked to government incentive schemes (payments to beneficiaries and FLWs), which could be a confounding factor.

• Also recently conducted analysis of the supply chain in Bihar found that supply of vaccines was erratic and irregular, which would also influence outcomes.

Looking forward

• Further research in understanding when frontline health workers use Mobile Kunji, with which women, and what the quality of that engagement looks like is necessary.

• It would also be invaluable to understand the impact of BBC Media Action’s interventions with frontline health workers (as opposed to households) on their ability to persuade families to adopt healthy behaviors.

• The results of BBC Media Action’s Usage & Engagement study, carried out by IMRB, should shed useful light on different Mobile Kunji use case scenarios and the impact of Mobile Kunji and Academy on changes in knowledge, attitudes, self-efficacy, social norms and specific practices. The analysis should be completed by the New Year.  

SHOPS is funded by the U.S. Agency for International Development.Abt Associates leads the project in collaboration withBanyan GlobalJhpiegoMarie Stopes InternationalMonitor GroupO’Hanlon Health Consulting

m4RH Impact Evaluation

Pamela Riley, Senior mHealth AdvisorDecember 11, 2014 Global mHealth Forum

Context

• Family planning protects the lives of women and children• Informed choice requires access to accurate, trusted

information about contraceptive options

• Mobiles provide a promising channel to promote demand for family planning services • SMS particularly well-suited to sensitive topics like

sexual health: personal, private, anonymous

• But what impact do SMS have?• Can they improve program outcomes?

m4RH overview

• USAID funded, developed by FHI360, in response to• Pervasive myths• Lack of info on family planning

• Free, user-initiated SMS service

• Partner support• MOH, Text to Change, NGOs

• Available in Kenya, Tanzania, Rwanda

Content

• Facts on 9 methods• Benefits• Side effects • Misconceptions • Clinic locator• Role model stories

• Installment 1: Hi, I’m John. I'm a 29 year-old university student. Yesterday my girlfriend says she wants to use family planning. I don’t know what this means and I’m embarrassed to ask her.  My friend Tupa says if I support Amina, she will go fishing and I will lose her.

Methodology

Study design

Research question: What is the impact of accessing m4RH service on knowledge and on use of contraceptive methods?

Sample size:13,629 new users

Data collection: • Surveys conducted via SMS• Nov 2013 to May 2014

Randomized controlled trial:

Treatment:

Full access to m4RH content

New mobile # accesses m4RH

Control: General motivational messages, clinic locations

Design considerations

• SMS data collection• Balancing depth of questions and response rate

against convenience of channel

• Control group• Denying m4RH access while engaging in study

• Timeline• 3 sets of 6 questions sent over a 90 day period

Data collection

• Response rate was substantial • 47% (6,432) answered

at least one knowledge question

• 24% (3,277) answered at least one use/behavior questions

• Similar response rates for treatment and control groups

Round 1(Same day)

Round 2(7 days)

Round 3(90 days)

0%

10%

20%

30%

40%

50%

60%

Survey Response by Round

Started but didn't completeCompleted

m4RH user profile

Average age is 25

Impla

ntsIU

D

Permanent

Inje

ction

Pill EC

condoms

Natura

l FP

LAM

Role m

odel sto

ries

Clinic

locatio

ns0%

10%

20%

30%

Content Requested

Interest similar across methods and topics

Under

18 20 23 26 29 32 35 38 41 44 47 50

0%

2%

4%

6%

8%

10%

12%

Male

Female

32% male

Results

Significant impact detected on knowledge

• Sample survey questions• When is a woman most

likely to get pregnant?

• How many days after sex is EC pill effective?

• About how long does IUD last before it needs to be replaced?

Control Treat-ment

Impact

2.06 2.33 0.27***(0.055)

Total knowledge correct

No impact detected on contraceptive use

• Contraceptive use was high (81%) for both groups

• No difference in the information accessed by contraceptive users and non-users

Control Treat-ment

Impact

Discussed with partner 39.7% 42.6% 2.9

(2.1)

Visited clinic 72.8% 73.9% 1.1(1.8)

Use of contraception 81.1% 80.8% -0.3

(1.5)

Implications

• SMS surveys feasible for measuring outcomes

• SMS effective in increasing knowledge

• Lack of impact on contraceptive use• Reach non-users • Limitations of single

channel interventions

SHOPS is funded by the U.S. Agency for International Development.Abt Associates leads the project in collaboration withBanyan GlobalJhpiegoMarie Stopes InternationalMonitor GroupO’Hanlon Health Consulting

Thank you

Study authors: Doug Johnson, Pamela Riley, Randall Juras, Soonie Choi, Kelly L’Engle, Francis Okello

Pamela RileyPamela_Riley@abtassoc.com

www.shopsproject.org