measure what matters – best practices in pr ......the amount of conversations generated by bots,...

39
Measure What Matters: Best Practices in PR Measurement GMP PR Measurement Conference, Bucharest, Romania May 2015 Katie Delahaye Paine CEO Paine Publishing www.painepublishing.com @queenofmetrics [email protected]

Upload: others

Post on 06-Jul-2020

0 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Measure What Matters – Best Practices in PR ......The amount of conversations generated by bots, spammers and pay-40% per-click sites 10% The percent of on-line conversations that

Measure What Matters: Best Practices in PR Measurement GMP PR Measurement Conference, Bucharest, Romania

May 2015 Katie Delahaye Paine CEO Paine Publishing

www.painepublishing.com

@queenofmetrics

[email protected]

Page 2: Measure What Matters – Best Practices in PR ......The amount of conversations generated by bots, spammers and pay-40% per-click sites 10% The percent of on-line conversations that

Paine Publishing: Providing communications

professionals the knowledge and information they need to navigate the journey to good measurement

Newsletters

Training Courses

Consulting

Katie Delahaye Paine: Helping communications professionals define and measure success for 25 years.

Founder of: The Delahaye Group KDPaine & Partners Paine Publishing

Author of: Measuring the Networked

Non-Profit Measure What Matters Measuring Public

Relationships

Page 3: Measure What Matters – Best Practices in PR ......The amount of conversations generated by bots, spammers and pay-40% per-click sites 10% The percent of on-line conversations that

It’s the end of the world as we knew it

The average audience for a MyDrunkKitchen video (http://www.youtube.com/watch?v=uSXQNred3is 500,000

CNN average nightly audience 179,000

The number of times per hour Digital Natives switch media—every 2.2

minutes. 27

The percent of conversation that happens OFF LINE 90%

The amount of conversations generated by bots, spammers and pay-

per-click sites 40%

The percent of on-line conversations that are public 10%

The percent of Facebook & Twitter posts that are actually seen < 5%

Page 4: Measure What Matters – Best Practices in PR ......The amount of conversations generated by bots, spammers and pay-40% per-click sites 10% The percent of on-line conversations that

Don’t Measure What’s Easy

4

Cartoon by Rob Cottingham

Page 5: Measure What Matters – Best Practices in PR ......The amount of conversations generated by bots, spammers and pay-40% per-click sites 10% The percent of on-line conversations that

Measure What Matters

27/05/2015 News Group International 5

Cartoon by Rob Cottingham

Page 6: Measure What Matters – Best Practices in PR ......The amount of conversations generated by bots, spammers and pay-40% per-click sites 10% The percent of on-line conversations that

What’s Changed?

Growth of media everywhere

Intolerance for messaging

It’s not about the media, it’s about your business & what your customers do with the information when you put it out there

The Barcelona Principles

6

Page 7: Measure What Matters – Best Practices in PR ......The amount of conversations generated by bots, spammers and pay-40% per-click sites 10% The percent of on-line conversations that

The Barcelona Principles, The Conclave & Industry Standards

1. Importance of Goal Setting & Measurement

2. Measuring the Effect on Outcomes is Preferred to Measuring Outputs

3. The Effect on Business Results Can & Should Be Measured Where Possible

4. Media Measurement Requires Quantity & Quality

5. Earned Media Value/AVEs are not the value of Public Relations

6. Social Media Can & Should be Measured

7. Transparency & Replicability are Paramount to Sound Measurement

All standards are available on

http://www.painepublishing.com/

standards-central

The Conclave

Content Sourcing & Methods 1

Reach and Impressions 2

Engagement 3

Influence & Relevance 4

Opinion & Advocacy 5

Impact & Value 6

Page 8: Measure What Matters – Best Practices in PR ......The amount of conversations generated by bots, spammers and pay-40% per-click sites 10% The percent of on-line conversations that

We need a new Attribution Model

ROI

Other Paid

Marketing

Digital/On-line Media

Buy

Print/TV Media

Buy

ROI

Media Relations/

Social Media/PR

Digital/ Online Media

Buy

Print/TV

Media Buy

Media

Relations/

Social

Media/PR

Page 9: Measure What Matters – Best Practices in PR ......The amount of conversations generated by bots, spammers and pay-40% per-click sites 10% The percent of on-line conversations that

Why Measuring Inches & Impressions IS NOT Value. Would you PAY for coverage like this?

“$1.1 Billion in Media Value”

Page 10: Measure What Matters – Best Practices in PR ......The amount of conversations generated by bots, spammers and pay-40% per-click sites 10% The percent of on-line conversations that

Eyeball counting

HITS Outcomes

MSM

Online

Social Media

Impressions are not awareness. Where’s the “So What?”

10

Page 11: Measure What Matters – Best Practices in PR ......The amount of conversations generated by bots, spammers and pay-40% per-click sites 10% The percent of on-line conversations that

Moving stakeholders up the engagement ladder

11

Advocacy Commitment Trial/Consideration

Followers Likes Impressions

Page 12: Measure What Matters – Best Practices in PR ......The amount of conversations generated by bots, spammers and pay-40% per-click sites 10% The percent of on-line conversations that

6 steps to standards-compliant measurement Step 1: Define your goal(s).

What outcomes is this strategy or tactic going to

achieve?

What are your measurable objectives?

Step 2: Define the parameters.

Who are you are trying to reach? How do your

efforts

connect with those audiences to achieve the goal?

Step 3: Define your benchmarks.

Who or what are you going to compare your results

to?

Step 4: Define your metrics.

What are the indicators to judge your progress?

Step 5: Select your data collection tool(s).

Step 6: Analyze your data.

Turn it into action, measure again

12

6 Steps

to Success

1

2

3

4

5

6

Page 13: Measure What Matters – Best Practices in PR ......The amount of conversations generated by bots, spammers and pay-40% per-click sites 10% The percent of on-line conversations that

Step 1: Define the goals

What return is expected? – Define in terms of the mission.

Define your champagne moment. – If you are celebrating complete 100% success a year from now, what is different about the organization?

13

Page 14: Measure What Matters – Best Practices in PR ......The amount of conversations generated by bots, spammers and pay-40% per-click sites 10% The percent of on-line conversations that

Goals, Actions, and Metrics

14

Goal

Increase in

preference for your destination

Increase heads in

bed

Action

Create events and campaigns that

increase preference

Use social media to

increase public engagement

Outcome Metric

% increase in target audience expressing preference brand

% increase in heads in beds

Activity Metric

% of quality conversation % increase in share of

desirable conversations

% increase in

engagement index

Page 15: Measure What Matters – Best Practices in PR ......The amount of conversations generated by bots, spammers and pay-40% per-click sites 10% The percent of on-line conversations that

Step 2: Understand the parameters What are management’s priorities?

Who are you are trying to reach?

How do your efforts connect with those audiences to achieve the goal?

What influences their decisions?

What’s important to them?

What makes them act?

15

Goal: Get the cat to stop howling

Options: Local? Cheap? Convenient?

Strategy: Buy cat food

Page 16: Measure What Matters – Best Practices in PR ......The amount of conversations generated by bots, spammers and pay-40% per-click sites 10% The percent of on-line conversations that

Step 3: Establish benchmarks

Past performance over time

Think 3

Whatever keeps your C-suite up at night

16

Page 17: Measure What Matters – Best Practices in PR ......The amount of conversations generated by bots, spammers and pay-40% per-click sites 10% The percent of on-line conversations that

Step 4: Pick your Kick-Butt Index

The Perfect KBI

Is actionable

Is there when you need it

Continuously improves your processes & gets you where you want to go

You become what you measure, so pick your KBI carefully

17

Page 18: Measure What Matters – Best Practices in PR ......The amount of conversations generated by bots, spammers and pay-40% per-click sites 10% The percent of on-line conversations that

Definitions of “Success” Workshop defined the criteria

All criteria linked back to the goals:

High quality media coverage

Intent to visit

Improved reputation

Visits

Page 19: Measure What Matters – Best Practices in PR ......The amount of conversations generated by bots, spammers and pay-40% per-click sites 10% The percent of on-line conversations that

Defining High Quality Coverage The OCS Score

Desirable Criteria Score Undesirable Score Score

Positive sentiment 1 Negative Sentiment -2

Contains one or More Positive

Messages 3

Contains one or more

Negative Messages -3

Event/Program is mentioned 2 No Event/Program is

mentioned 0

Appears in Tier1 Media 2 Negative Mention in Tier1 -1

Third Party Endorsement 1 Recommends competition -2

Contains a desirable visual 1 Contain undesirable visual -2

Total Score 10 Total -10

Page 20: Measure What Matters – Best Practices in PR ......The amount of conversations generated by bots, spammers and pay-40% per-click sites 10% The percent of on-line conversations that

Social Media Engagement Index

Action Score

Like/Follow/Opens/+1 .5

Favorite or Opens or Views 1

Comment 1.5

Share content 2

Signs up to receive email or other owned content 2.5

Shares a link to an owned site 2.5

Total 10

24

Page 21: Measure What Matters – Best Practices in PR ......The amount of conversations generated by bots, spammers and pay-40% per-click sites 10% The percent of on-line conversations that

Step 5: Selecting a measurement tool

21

Objective KBI Tool

Increase message communications

Increase percent of items containing one or more messages

Media Content Analysis

(CyberAlert, Prime, Carma)

Increase

awareness/

preference

% of audience preferring your brand

to the competition

Survey Monkey

Engage

marketplace

% increase in engagement on website and/or social sites

Unmetric, Simply

Measured, Google Analytics, Site Catalyst,

Network Analysis

Page 22: Measure What Matters – Best Practices in PR ......The amount of conversations generated by bots, spammers and pay-40% per-click sites 10% The percent of on-line conversations that

All the tools you really need

Google Analytics

Ignore page views

Set up conversions

Excel

Survey Monkey

5/27/2015

Page 23: Measure What Matters – Best Practices in PR ......The amount of conversations generated by bots, spammers and pay-40% per-click sites 10% The percent of on-line conversations that

Step 6: Be Data Informed, not Data Driven

Rank order results from worst to best

Ask “So What?” at least three times

Put your data into an overall framework consistent with C-Suite expectations

Find your “Data Geek”

Compare to last month, last quarter, 13-month average

23 Page 23

Page 24: Measure What Matters – Best Practices in PR ......The amount of conversations generated by bots, spammers and pay-40% per-click sites 10% The percent of on-line conversations that

Photo Event

High Quality Content

Resource Use

Low

Hig

h

Med

ium

Ver

y hi

gh

Level of Engagement

Ver

y H

igh

Med

ium

Hig

h

Low

High Resources

Low Quality Content

Low Resources

Webinar

Status update

Link

Ultimate

Road Trip

Google + Chat

Media Day

Corporate Video

What gets the most bang for the buck?

Resource Use

Low

Hig

h

Med

ium

Ver

y hi

gh

Ver

y H

igh

Med

ium

Hig

h

Low

Webinar

Status update

Link

Ultimate

Road Trip

Google + Chat

Media Day

Corporate Video

Page 25: Measure What Matters – Best Practices in PR ......The amount of conversations generated by bots, spammers and pay-40% per-click sites 10% The percent of on-line conversations that

Case Studies

Page 26: Measure What Matters – Best Practices in PR ......The amount of conversations generated by bots, spammers and pay-40% per-click sites 10% The percent of on-line conversations that

NATO Measurement Program Parameters Goal: Move from quantity-based metrics to quality-

based metrics

Began in October 2014

Used January-May 2015 as benchmark

Customized “Quality Score” that indexes message content, visuals, quotes, sentiment, and media importance on a scale of +10 to -10

Raw data is collected and coded on Top Tier media only.

Page 27: Measure What Matters – Best Practices in PR ......The amount of conversations generated by bots, spammers and pay-40% per-click sites 10% The percent of on-line conversations that

Only 2 out of 4 Messages are Getting Traction

140 131

115

36

157

52

31

1

17

2

1

Jan Feb Mar Apr May

Capable & adaptive assurance parternships values

Page 28: Measure What Matters – Best Practices in PR ......The amount of conversations generated by bots, spammers and pay-40% per-click sites 10% The percent of on-line conversations that

Spokespeople Drive Messages

169

252

87

154

2

17

1 no

yes

Spokespeople Drive Messages

Capable & adaptive assurance Partnerships values - (blank)

Page 29: Measure What Matters – Best Practices in PR ......The amount of conversations generated by bots, spammers and pay-40% per-click sites 10% The percent of on-line conversations that

ACA Measurement Program Parameters Goal: Move to Industry Standards Metrics and away from

media value and ad value equivalency

Measure quality not just quantity

Began in January 2013

Used January-March 2013 as benchmark

Customized “Quality Score” that indexes message content, tone, and media importance on a scale of +10 to -10

Correlated with web traffic to measure outcomes

Data is available on demand as well as in quarterly reports

Raw data is collected and coded on Top Tier media only

Data is processed through a custom algorithm to enable analysis and derive OCS Scores

Page 30: Measure What Matters – Best Practices in PR ......The amount of conversations generated by bots, spammers and pay-40% per-click sites 10% The percent of on-line conversations that

Negative Coverage of AC is Constant & Growing

0

100

200

300

400

500

600

700

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun

Tone of Coverage about Atlantic City Over Time

Negative Neutral Positive

Miss America

Page 31: Measure What Matters – Best Practices in PR ......The amount of conversations generated by bots, spammers and pay-40% per-click sites 10% The percent of on-line conversations that

Without ACA events, OCS Scores for Atlantic City would have been significantly lower

4

3.25

2.75 2.99

3.65

2.96 3.36 3.24

2.34 2.37 2.43

1.30

-1.24

0.37 -0.05

0.28 0.28

-1.56

4.91

3.92

2.99

3.58

4.14 4.1 4.27 4.12 4.29

2.78 2.56

1.53

-0.29

0.61 0.20

1.44 1.77

0.63

-2

-1

0

1

2

3

4

5

6

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May June

Ave

rage

OC

S Sc

ore

Average OCS Score Over Time

Atlantic City OCS without ACA Atlantic City OCS with ACA

The red line represents coverage of Atlantic City minus all mentions of ACA and its programs

Page 32: Measure What Matters – Best Practices in PR ......The amount of conversations generated by bots, spammers and pay-40% per-click sites 10% The percent of on-line conversations that

ACA has made Demonstrable Difference in Quality of Media Coverage

If ACA or one of its programs were mentioned in a media story, it was:

More likely to contain an endorsement

More likely to be positive

More likely to contain key messages

Less likely to be negative

0.22

0.00

0.62

1.42

4.61

0.08

0.08

0.01

-0.61

-0.66

Endorsements

Positive Visual

Negative Visual

Tone

MessageCommunication

Average of OCS

Differences in Quality of Coverage with & without ACA

Atlantic City Atlantic City Alliance

Page 33: Measure What Matters – Best Practices in PR ......The amount of conversations generated by bots, spammers and pay-40% per-click sites 10% The percent of on-line conversations that

ACA Maintains High OCS Score with Proactive Program Coverage

Lady Antebellum, Non-Gambling Fun,

Hello Summer, GMA segment

14 44 31

93 87 75 75

40

95

28 7

27 17 18 12

107 122

254

5.5

7.18

6.3

5.02

6.24 6.42

7.42 7.21

6.54

5.27

4.25

5.08

6.47

5.94

5.33

4.75

6.87

6.41

0

1

2

3

4

5

6

7

8

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May June

0

50

100

150

200

250

300

OC

S Sc

ore

Nu

mb

er o

f m

enti

on

s

ACA Mentions vs. OCS Over Time

ACA Mentions ACA OCS

Miss America, ACA new

campaign, Blake Shelton, World Poker

Championship

Page 34: Measure What Matters – Best Practices in PR ......The amount of conversations generated by bots, spammers and pay-40% per-click sites 10% The percent of on-line conversations that

Communication of the "Making a Comeback" & "Something for Everyone" messages increased as ACA

announced new programs

14 39

23 36

133

235

8 7 4 2 10 15 8 11 1 2 9 2 24

70

21

73 88

110

13

104 74

138

87

48

Jan Feb Mar Apr May Jun

Qtr1 Qtr2

2014

Atlantic City has something for everyone

Atlantic City is a clean and safe place to visit

Atlantic City is a year-round destination spot

Atlantic City is making a comeback

Atlantic City offers more than just gaming

Page 35: Measure What Matters – Best Practices in PR ......The amount of conversations generated by bots, spammers and pay-40% per-click sites 10% The percent of on-line conversations that

When ACA Programs Received Media Coverage, Traffic Followed

0

50

100

150

200

250

300

0

20000

40000

60000

80000

100000

120000

140000

Relationship between ACA Program Mentions and Site Visits

Site Visits Program Mentions

Page 36: Measure What Matters – Best Practices in PR ......The amount of conversations generated by bots, spammers and pay-40% per-click sites 10% The percent of on-line conversations that

ACA Programs Drive Higher OCS Scores, which correlate highly with web visits

0.41

0.44

0.47

AC items

ACA Items

ACA OCS Scores

Correlations between Web Visits and PR Metrics

Pearson r. value

Page 37: Measure What Matters – Best Practices in PR ......The amount of conversations generated by bots, spammers and pay-40% per-click sites 10% The percent of on-line conversations that

PR is having a positive impact on preference & perceptions

Based on the results of the most recent brand tracking survey, it appears that people who say they have recently heard news about Atlantic City are very likely to perceive AC as fun – a key driver of preference.

Respondents who remember seeing news reports about Atlantic City are also very likely to associate key messages and positioning statements such as “place I am excited to go to” and “fun place to hang out with friends.”

The same respondents are also more likely to recommend Atlantic City to friends.

The largest percentage of respondents who remember seeing news about Atlantic City don’t remember where they saw it. If they do remember, most saw such news in newspapers and online media.

Page 38: Measure What Matters – Best Practices in PR ......The amount of conversations generated by bots, spammers and pay-40% per-click sites 10% The percent of on-line conversations that

Best Practices in Measurement Start with SMART Objectives

Get agreement on role that PR plays in path to purchase

Agree on what criteria are critical, i.e.

Dispelling a myth

Desirable visuals

Recommendations

Stories that leave the reader more likely to visit

Don’t use measurement to justify your budgets, use it to make strategic decisions

Page 39: Measure What Matters – Best Practices in PR ......The amount of conversations generated by bots, spammers and pay-40% per-click sites 10% The percent of on-line conversations that

Thank You!

For more information on measurement, visit: www.PainePublishing.com

Subscribe to our bi-monthly newsletter: The Measurement Advisor

Email me: [email protected]

Follow me on Twitter: @queenofmetrics

Or call me: 1-603-682-0735

39