so what about that behaviour change?2018.wsmconference.co.uk/wp-content/uploads/2014/... · example...
TRANSCRIPT
Over 14 million views within a week of posting the video to YouTube.
Impressions are useful measure for exposure, but the objective of a safety campaign is behaviour change, not impressions or click-throughs or Facebook shares.
So what about that behaviour change?
Leah Waymark, the General Manager of
Corporate Relations at Metro Trains,
summarizes the evaluation criteria as
“…if we can save one life or avoid
serious injury, then that’s how we’ll
measure the success of this
campaign”
I can’t find anything elsewhere that describes
the campaign’s evaluation strategy.
Workshop 2:
Selecting and setting
Behavioural Goals, and
Social Marketing planning
Drs Julie Huibregtsen and
Professor Jeff French
“Social Marketing is about
creating something that is
valued by your audience”
What is Sociale Marketing?
= citizen based
© Julie Huibregtsen, Huibregtsen Training & Advies
FREE HEALTHY breakfast club
Music ClubTV, music & celebrity magazines
+ a healthy breakfast
‘Eating healthy, before
entering school’ Passing the doorstep, download some music
and eat some sandwiches, while having ‘fun with friends’© Julie Huibregtsen, Huibregtsen Sociale marketing
Social Marketing
• Integrated Intervention Mix
• Co-creation through Social Markets
• Competition analysis and Action
• Systematic planning & evaluation
• Insight driven Segmentation
Social
Marketing Techniques
• BehavioralInfluence
• Citizen/ Orientation Focus)
• Social Offerings (Idea, product, service experience
• Relationship building
Social Marketing Concepts
• Social Value Creation through
Exchange and Social Offerings( Idea, product, service experience
The Social Marketing Principle
The big frustrating questions for
Donors and Governments
What is the impact
of the funds we
invest?
What is the ROI?
What have we
learnt?
Lord Kelvin
26 June 1824 – 17 December
1907
To measure is to
know
Knowledge is
power
Francis Bacon
22 January 1561 – 9 April
1626
If you can’t measure
it you can’t manage
it.
What gets
measured gets
done
Social Marketing PlanningSocial Marketing planning models need to be applied alongside the concepts and principles set out above. The four step STELa model is one such planning process.
http://www.stelamodel.com/
The Rationale
Situation Analysis
Target Audience Profile
Intervention Proposition
Initial Marketing
Objectives
Marketing
Intervention Mix
Strategies
Pre -testing and
Piloting
Report on the Pilot
Programme
Full Business Plan
Time Frame and Key
Mile stones
Resources Allocation
Stakeholder and
Partner Management
Evaluation and
Monitoring
Reporting
Dissemination
Review and Plan to
Build on Learning
and Evaluation
Test
Learn
&
ActScope Test Enact Learn Act
French 2010
Step 1 Setting aims and objectives
AIM: a broad strategic purpose of a project, AIMS can be long term, medium term or short term.
Objective: a specific measurable goal whose achievement will contribute towards the aim
Why so important?
1.Set clear behavioural goals, everyone (including
yourself) knows what you want to achieve and are
asking of your target audiance
2.You can messure it
3.You can define what the barriers and benefits are
(because it’s tangible)……
4.It ‘forces’ you to start with the target audience and
determine what the current behaviour is and the
desired behaviour must be
Focus on behaviour
Set a behavioural goal:
2 % less obesity is not a behavioural
goal,taking the stairs is……………
Better awareness of environment isn’t,
recycling and so really throwing away your
glass away is….
© Julie Huibregtsen, Huibregtsen Training & Advies
Example of desired
behaviour: Fun Theory
© Julie Huibregtsen, Huibregtsen Training & Advies
Specific: Not open to different interpretations
Measurable: Can observe & collect objective measures
Achievable: Not attempting too much
Realistic: Given human & financial resources
Time bound: Time scales / stages clearly identified
SMART objectives
Objectives can be divided into:
Affective objectives, focused of feelings / emotion
Cognitive objectives, focused on learning / knowing
Psychomotor objectives, focused on doing/ behaviour
Objectives can be divided into:
Affective objectives, focused of feelings / emotion
Cognitive objectives, focused on learning / knowing
Psychomotor objectives, focused on doing/ behaviour
How to use behavioural
objectives to evaluate social
marketing programmes
It happens every dayColleagues,
I recently ran a multi-channel media communications campaign for promoting colon cancer screening (CCS). I would have preferred an all-inclusive social marketing approach, but time, budget and the small ‘p’ political considerations happened.
One significant component left out of the mix was evaluation. At the time of our initial planning the team had neither funding nor an epidemiologist/evaluator. As the end of the fiscal year draws near, additional funds have been identified for evaluation.
I’ve been asked to supply ‘evidence’ or citations to support my arguments I would appreciate any feedback you are willing to provide.
Thanks Jo
Evaluation
• Judgement based on careful
assessment and critical
appraisal (WHO 1981)
• Making a judgement about the
value of something by looking
critically (Ewles and Simnett
1995)
Evaluation can be defined as:
• A process which sets out to determine
systematically and objectively the,
effectiveness, efficiency and
relevance of activities based on
specified objectives. Evaluation is a
process for improving current
activities, future programmes.
Evaluation can also aid decision-
making about resource allocation.
Invisible Evaluation In Victoria AUSTRALIA , during 2001, the
Government allocated $ 6 MILLION to a new
advertising and communications campaign to
reduce problem gambling.
IN 2002 Calls to the problem gambling helpline
increased by 70% and face-to face clients
increased by 118%.
The Department’s 2002-2003 Annual
Report stated that an evaluation of the
campaign had been funded and was in
progress, however, the results of
this publication remain
unavailable
1. To assess the extent to which projects are
achieving their aims and objectives
2. To inform the developments of materials,
methods and other interventions
3. To ensure ethical practice
4. To optimise use of resources
9 reasons for evaluation
9 reasons for evaluation
5. To inform programme planning
6. To inform practice and contribute tothe evidence base
7. Provide funders with evidence of success
8. Provide feedback for those involved
9. Help build an performance culture
4 step Evaluation
1. Setting Aims and Objectives
2. Identifying Evaluation Indicators
3. Choosing data collection methods
4. Dissemination and Action
What is a metric?
A set of numbers that
indicate that the objective
you have set is being
achieved
What kinds of measures
or metrics can we use to
measure
social marketing
programmes?
Discuss
What Makes a good metric?
1. Rational
2. Relevant
3. Valid
4. Comparable
5. Timely
6. Scalable
7. Easily explained and communicated
8. Not induce perverse incentives
or unintended consequences
You have identified a perfect metric when:
1.Nobody complains about it!
2. Users say it helps them choose services & builds confidence
3. Colleagues say it empowers and encourages them
4. Managers use it to improve efficiency
5.Politicians ask you to develop some more!
EvaluationFormative Evaluation: Indicators e.g.: Did we have the
right data, did we use the right theory and evidence, did we set the
right objectives.
Process Evaluation: Indicators: e.g.: How many people took
part in or took advantage of a service that was offered, how many people
were contacted.
Impact Evaluation: Indicators: e.g.: Data on immediate impact
of the work. For example: reported increase in knowledge, reported
behaviour or an immediate increase in the take up of a service.
Outcome Evaluation: Indicators: e.g.: Data on the longer
term effects of the project, observed behaviour and its consequences. For
example: less obese people or more recycling.
Cost Per Lead
CPL = TPB
TLG
CPL = Cost Per Lead
TPB = Total
Programme Budget
TLG = Total Leads
Generated
Cost Per Change
CPC = TPB
NODBC
CPC = Cost Per Change
TPB = Total Programme Budget
NODC = Number of Desired Behaviour Changes
£100,00
1000 = £100
ROI(Return On Investment)
The ratio of £ returned for every £ spent, once
the activity has paid for itself e.g.
£100,000 Spent
£500,000 Gained
£400,000 Net
Ratio of 1:4
Example estimates of ROI on a proposal to extend smoking cessation clinic opening times
Numbers of current attendance annually 1,000
Percentage increase projected by increasing opening times
10%
Number of new clients 100
Percentage of new clients expected to quit 25%
Number of additional quitters 25
Average cost saving per quitter £3,000
Annual gross cost saving (25 x £3,000) £75,000
Annual costs of providing extended opening £20,000
Net cost savings £55,000
ROI (55,000/20,000) 2.75 : 1 or 275%
ROI
Insight driven
Evidence driven
Consistent
Sustained
Segmented approach
THINK! CAMPAIGN SUCCESS
ROI as social ‘value’ Saved over 1000 lives and counting
Prevented 90,000 injuries and counting
Represents 3,494 people (and counting) who are alive and uninjured today who wouldn’t have been without THINK!
The monetary value to society was £4.2billion.
It is estimated that, for every £1 spent on
THINK! the campaign has saved society £9.36
54
Educational, enforcement and
engineering to improve road
safety.
Pre Think! total casualties grew
by 3%, and total accidents by
2%. Since Think! total casualties
and accidents declined by 12%
and 11% respectively.
For more information see:
www.thinkroadsafety.gov.uk
A sustained programme
Cost Benefit Analysis
The cost of the problem V the cost
savings from the intervention
PaybackGenerating economic value e.g. recruitment of the best teachers improves education and economic output
Generating revenue e.g. a campaign to encourage people to inform on tax avoiders
Saving public money directly e.g. encouraging more people to complete tax returns online
Saving public money indirectly e.g. smoking campaigns reducing the cost on the NHS
Spending public money well e.g., reducing administration Costs
Value For Money' (VFM)
Assesses if an organisation has obtained the
maximum benefit from the resources it controls.
Some elements may be subjective, difficult to
measure, and intangible. Judgements are required.
Takes account of : quality, cost, resources
used, fitness for purpose, timeliness, and
convenience to judge whether or not,
together, they constitute good value.
ROI and VFM
100% Effective
100% Efficient
100% Quality
Compliant
0% Effective
0% Efficient
0% Quality
Compliant
Range
of
realistic
impact
Range
of
realistic
impact
Learning and
continuous
improvement is the
major reason for
identifying and
using performance
metrics
Modified from Ralph D. Stacey: "Complexity and Creativity in Organizations"
When to use SMART Objectives and
when to use more general Goals
Modified from Ralph D. Stacey: "Complexity and Creativity in Organizations"
Modified from Ralph D. Stacey: "Complexity and Creativity in Organizations"
Modified from Ralph D. Stacey: "Complexity and Creativity in Organizations"
[email protected]: Stacey RD. Strategic management and organisational
dynamics: the challenge of complexity. 3rd ed. Harlow: Prentice Hall,
2002.
SMART
OBJECTIVES AND
PRESCRIBED
SYSTEMS AUDIT
MORE
GENERAL
GOALS AND
EVALUATION
When to use tight ‘Objectives’
When to use more general ‘Goals’
When there is:
Strong evidence & consensus, a high degree of
specification of objectives and methodology is most
effective.
When there is :
Less consensus on what works the best approach
is to set general goals and foster evaluation and
research to feed the evidence base.
Why is ownership of evaluation and
behaviour goals important for
organisations ?
1. Self assessment creates ownership
and can be a powerful tool for
improvement
2. Without ownership target fixing and
sabotage is possible
Involving recipients in evaluation
1. Helps to ensure that the project is meeting needs
2. Strengthens community ownership of the project
3. Helps to build community capacity
4. Can contribute to the sustainability of the project
Metrics must
align with
Strategic Goals
and be
Simple enough that
each person sees that
their work contributes
to achieving the
selected metrics.
SM a good buy ? YESAn Ounce of Prevention – what are the returns? (2nd
Edn 1999) CDC - USAftp://ftp.cdc.gov/pub/Publications/mmwr/other/ozprev.pdf
Returns on Investment in Public Health: An
epidemiological and economic analysis (2003) Access
Economics - Australiahttp://www.health.gov.au/internet/wcms/publishing.nsf/Content/health-pubhlth-publicat-document-roi_eea-cnt.htm
Securing good health for the whole population: Final
report - February 2004http://www.dh.gov.uk/en/PublicationsAndStatistics/Publications/PublicationsPolicyAndGuidance/DH_4074426
CDC Community Programmehttp://www.cdc.gov/healthycommunitiesprogram/
National Institute for Health and Clinical Excellencehttp://www.sussedprofessionals.net/files/PH006quickrefguide.pdf
