measure what matters – best practices in pr ......the amount of conversations generated by bots,...
TRANSCRIPT
Measure What Matters: Best Practices in PR Measurement GMP PR Measurement Conference, Bucharest, Romania
May 2015 Katie Delahaye Paine CEO Paine Publishing
www.painepublishing.com
@queenofmetrics
Paine Publishing: Providing communications
professionals the knowledge and information they need to navigate the journey to good measurement
Newsletters
Training Courses
Consulting
Katie Delahaye Paine: Helping communications professionals define and measure success for 25 years.
Founder of: The Delahaye Group KDPaine & Partners Paine Publishing
Author of: Measuring the Networked
Non-Profit Measure What Matters Measuring Public
Relationships
It’s the end of the world as we knew it
The average audience for a MyDrunkKitchen video (http://www.youtube.com/watch?v=uSXQNred3is 500,000
CNN average nightly audience 179,000
The number of times per hour Digital Natives switch media—every 2.2
minutes. 27
The percent of conversation that happens OFF LINE 90%
The amount of conversations generated by bots, spammers and pay-
per-click sites 40%
The percent of on-line conversations that are public 10%
The percent of Facebook & Twitter posts that are actually seen < 5%
Don’t Measure What’s Easy
4
Cartoon by Rob Cottingham
Measure What Matters
27/05/2015 News Group International 5
Cartoon by Rob Cottingham
What’s Changed?
Growth of media everywhere
Intolerance for messaging
It’s not about the media, it’s about your business & what your customers do with the information when you put it out there
The Barcelona Principles
6
The Barcelona Principles, The Conclave & Industry Standards
1. Importance of Goal Setting & Measurement
2. Measuring the Effect on Outcomes is Preferred to Measuring Outputs
3. The Effect on Business Results Can & Should Be Measured Where Possible
4. Media Measurement Requires Quantity & Quality
5. Earned Media Value/AVEs are not the value of Public Relations
6. Social Media Can & Should be Measured
7. Transparency & Replicability are Paramount to Sound Measurement
All standards are available on
http://www.painepublishing.com/
standards-central
The Conclave
Content Sourcing & Methods 1
Reach and Impressions 2
Engagement 3
Influence & Relevance 4
Opinion & Advocacy 5
Impact & Value 6
We need a new Attribution Model
ROI
Other Paid
Marketing
Digital/On-line Media
Buy
Print/TV Media
Buy
ROI
Media Relations/
Social Media/PR
Digital/ Online Media
Buy
Print/TV
Media Buy
Media
Relations/
Social
Media/PR
Why Measuring Inches & Impressions IS NOT Value. Would you PAY for coverage like this?
“$1.1 Billion in Media Value”
Eyeball counting
HITS Outcomes
MSM
Online
Social Media
Impressions are not awareness. Where’s the “So What?”
10
Moving stakeholders up the engagement ladder
11
Advocacy Commitment Trial/Consideration
Followers Likes Impressions
6 steps to standards-compliant measurement Step 1: Define your goal(s).
What outcomes is this strategy or tactic going to
achieve?
What are your measurable objectives?
Step 2: Define the parameters.
Who are you are trying to reach? How do your
efforts
connect with those audiences to achieve the goal?
Step 3: Define your benchmarks.
Who or what are you going to compare your results
to?
Step 4: Define your metrics.
What are the indicators to judge your progress?
Step 5: Select your data collection tool(s).
Step 6: Analyze your data.
Turn it into action, measure again
12
6 Steps
to Success
1
2
3
4
5
6
Step 1: Define the goals
What return is expected? – Define in terms of the mission.
Define your champagne moment. – If you are celebrating complete 100% success a year from now, what is different about the organization?
13
Goals, Actions, and Metrics
14
Goal
Increase in
preference for your destination
Increase heads in
bed
Action
Create events and campaigns that
increase preference
Use social media to
increase public engagement
Outcome Metric
% increase in target audience expressing preference brand
% increase in heads in beds
Activity Metric
% of quality conversation % increase in share of
desirable conversations
% increase in
engagement index
Step 2: Understand the parameters What are management’s priorities?
Who are you are trying to reach?
How do your efforts connect with those audiences to achieve the goal?
What influences their decisions?
What’s important to them?
What makes them act?
15
Goal: Get the cat to stop howling
Options: Local? Cheap? Convenient?
Strategy: Buy cat food
Step 3: Establish benchmarks
Past performance over time
Think 3
Whatever keeps your C-suite up at night
16
Step 4: Pick your Kick-Butt Index
The Perfect KBI
Is actionable
Is there when you need it
Continuously improves your processes & gets you where you want to go
You become what you measure, so pick your KBI carefully
17
Definitions of “Success” Workshop defined the criteria
All criteria linked back to the goals:
High quality media coverage
Intent to visit
Improved reputation
Visits
Defining High Quality Coverage The OCS Score
Desirable Criteria Score Undesirable Score Score
Positive sentiment 1 Negative Sentiment -2
Contains one or More Positive
Messages 3
Contains one or more
Negative Messages -3
Event/Program is mentioned 2 No Event/Program is
mentioned 0
Appears in Tier1 Media 2 Negative Mention in Tier1 -1
Third Party Endorsement 1 Recommends competition -2
Contains a desirable visual 1 Contain undesirable visual -2
Total Score 10 Total -10
Social Media Engagement Index
Action Score
Like/Follow/Opens/+1 .5
Favorite or Opens or Views 1
Comment 1.5
Share content 2
Signs up to receive email or other owned content 2.5
Shares a link to an owned site 2.5
Total 10
24
Step 5: Selecting a measurement tool
21
Objective KBI Tool
Increase message communications
Increase percent of items containing one or more messages
Media Content Analysis
(CyberAlert, Prime, Carma)
Increase
awareness/
preference
% of audience preferring your brand
to the competition
Survey Monkey
Engage
marketplace
% increase in engagement on website and/or social sites
Unmetric, Simply
Measured, Google Analytics, Site Catalyst,
Network Analysis
All the tools you really need
Google Analytics
Ignore page views
Set up conversions
Excel
Survey Monkey
5/27/2015
Step 6: Be Data Informed, not Data Driven
Rank order results from worst to best
Ask “So What?” at least three times
Put your data into an overall framework consistent with C-Suite expectations
Find your “Data Geek”
Compare to last month, last quarter, 13-month average
23 Page 23
Photo Event
High Quality Content
Resource Use
Low
Hig
h
Med
ium
Ver
y hi
gh
Level of Engagement
Ver
y H
igh
Med
ium
Hig
h
Low
High Resources
Low Quality Content
Low Resources
Webinar
Status update
Link
Ultimate
Road Trip
Google + Chat
Media Day
Corporate Video
What gets the most bang for the buck?
Resource Use
Low
Hig
h
Med
ium
Ver
y hi
gh
Ver
y H
igh
Med
ium
Hig
h
Low
Webinar
Status update
Link
Ultimate
Road Trip
Google + Chat
Media Day
Corporate Video
Case Studies
NATO Measurement Program Parameters Goal: Move from quantity-based metrics to quality-
based metrics
Began in October 2014
Used January-May 2015 as benchmark
Customized “Quality Score” that indexes message content, visuals, quotes, sentiment, and media importance on a scale of +10 to -10
Raw data is collected and coded on Top Tier media only.
Only 2 out of 4 Messages are Getting Traction
140 131
115
36
157
52
31
1
17
2
1
Jan Feb Mar Apr May
Capable & adaptive assurance parternships values
Spokespeople Drive Messages
169
252
87
154
2
17
1 no
yes
Spokespeople Drive Messages
Capable & adaptive assurance Partnerships values - (blank)
ACA Measurement Program Parameters Goal: Move to Industry Standards Metrics and away from
media value and ad value equivalency
Measure quality not just quantity
Began in January 2013
Used January-March 2013 as benchmark
Customized “Quality Score” that indexes message content, tone, and media importance on a scale of +10 to -10
Correlated with web traffic to measure outcomes
Data is available on demand as well as in quarterly reports
Raw data is collected and coded on Top Tier media only
Data is processed through a custom algorithm to enable analysis and derive OCS Scores
Negative Coverage of AC is Constant & Growing
0
100
200
300
400
500
600
700
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun
Tone of Coverage about Atlantic City Over Time
Negative Neutral Positive
Miss America
Without ACA events, OCS Scores for Atlantic City would have been significantly lower
4
3.25
2.75 2.99
3.65
2.96 3.36 3.24
2.34 2.37 2.43
1.30
-1.24
0.37 -0.05
0.28 0.28
-1.56
4.91
3.92
2.99
3.58
4.14 4.1 4.27 4.12 4.29
2.78 2.56
1.53
-0.29
0.61 0.20
1.44 1.77
0.63
-2
-1
0
1
2
3
4
5
6
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May June
Ave
rage
OC
S Sc
ore
Average OCS Score Over Time
Atlantic City OCS without ACA Atlantic City OCS with ACA
The red line represents coverage of Atlantic City minus all mentions of ACA and its programs
ACA has made Demonstrable Difference in Quality of Media Coverage
If ACA or one of its programs were mentioned in a media story, it was:
More likely to contain an endorsement
More likely to be positive
More likely to contain key messages
Less likely to be negative
0.22
0.00
0.62
1.42
4.61
0.08
0.08
0.01
-0.61
-0.66
Endorsements
Positive Visual
Negative Visual
Tone
MessageCommunication
Average of OCS
Differences in Quality of Coverage with & without ACA
Atlantic City Atlantic City Alliance
ACA Maintains High OCS Score with Proactive Program Coverage
Lady Antebellum, Non-Gambling Fun,
Hello Summer, GMA segment
14 44 31
93 87 75 75
40
95
28 7
27 17 18 12
107 122
254
5.5
7.18
6.3
5.02
6.24 6.42
7.42 7.21
6.54
5.27
4.25
5.08
6.47
5.94
5.33
4.75
6.87
6.41
0
1
2
3
4
5
6
7
8
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May June
0
50
100
150
200
250
300
OC
S Sc
ore
Nu
mb
er o
f m
enti
on
s
ACA Mentions vs. OCS Over Time
ACA Mentions ACA OCS
Miss America, ACA new
campaign, Blake Shelton, World Poker
Championship
Communication of the "Making a Comeback" & "Something for Everyone" messages increased as ACA
announced new programs
14 39
23 36
133
235
8 7 4 2 10 15 8 11 1 2 9 2 24
70
21
73 88
110
13
104 74
138
87
48
Jan Feb Mar Apr May Jun
Qtr1 Qtr2
2014
Atlantic City has something for everyone
Atlantic City is a clean and safe place to visit
Atlantic City is a year-round destination spot
Atlantic City is making a comeback
Atlantic City offers more than just gaming
When ACA Programs Received Media Coverage, Traffic Followed
0
50
100
150
200
250
300
0
20000
40000
60000
80000
100000
120000
140000
Relationship between ACA Program Mentions and Site Visits
Site Visits Program Mentions
ACA Programs Drive Higher OCS Scores, which correlate highly with web visits
0.41
0.44
0.47
AC items
ACA Items
ACA OCS Scores
Correlations between Web Visits and PR Metrics
Pearson r. value
PR is having a positive impact on preference & perceptions
Based on the results of the most recent brand tracking survey, it appears that people who say they have recently heard news about Atlantic City are very likely to perceive AC as fun – a key driver of preference.
Respondents who remember seeing news reports about Atlantic City are also very likely to associate key messages and positioning statements such as “place I am excited to go to” and “fun place to hang out with friends.”
The same respondents are also more likely to recommend Atlantic City to friends.
The largest percentage of respondents who remember seeing news about Atlantic City don’t remember where they saw it. If they do remember, most saw such news in newspapers and online media.
Best Practices in Measurement Start with SMART Objectives
Get agreement on role that PR plays in path to purchase
Agree on what criteria are critical, i.e.
Dispelling a myth
Desirable visuals
Recommendations
Stories that leave the reader more likely to visit
Don’t use measurement to justify your budgets, use it to make strategic decisions
Thank You!
For more information on measurement, visit: www.PainePublishing.com
Subscribe to our bi-monthly newsletter: The Measurement Advisor
Email me: [email protected]
Follow me on Twitter: @queenofmetrics
Or call me: 1-603-682-0735
39