Measurement 2.0-- Best Practices in Social Media measurement A presentation for AMEC Katie Delahaye PaineCEOkdpaine@kdpaine.comwww.measuresofsuccess.comhttp:/kdpaine.blogs.comMember, IPR Measurement Commissionwww.instituteforpr.org

A measurement timeline

The changing “Holy Grail” of measurement

Engagement

On your property?

With your brand?

Relationships

With what audience?

Must be competitive?

ROI

AVE does not equal ROI

ROI =Desired Return minus

Investment

The immutable laws of 21st Century PR Measurement

1. There is no market for your message2. All the benchmarks have changed3. Size doesn’t matter so stop screaming,

start listening4. It’s not how many eyeballs, it’s the

right eyeballs5. HITS = How Idiots Track Success6. ROI doesn’t mean what you think it

does because you can’t divide by zero7. You become what you measure, so

match the measurement tool to your objective

Signs that it’s the end of the world as we know it 8. BestBuy measures 85% lower turnover as a

result of its Blue Shirt community7. State Farm measures it’s internal blog by the

improvement in morale6. ASPCA can track on-line donations and

increased membership back to its social media efforts.

5. On Twitter, a start up company got 100 great marketing ideas for free, women raised over $6000 in a day and a wooden toy maker in NH got a nationwide contract

4. $0-budget YouTube videos about Barack Obama were seen by 120 times the audience of Hilary Clinton’s “largest town hall meeting in US history” that cost millions

3. IBM receives more leads, sales and exposure from a $500 podcast than it does from an ad

2. Advertisers are starting to admit that all their measures are flawed

1. Measurement is easy

What do you need to measure?Outputs?

Did you get the coverage you wanted?Did you produce the promised materials on

time and on budget? Outtakes?

Did your target audience see the messages?Did they believe the messages?

Outcomes?Did audience behavior change? Did the right people show up?Did your relationship change? Did sales increase?

Goals, Actions and Metrics Goal Action Output Metric Outtake

MetricOutcome Metric

Increased on-line reservations

Revamp website

Amount of content on web site

% perceiving state as a destination

% increase in web traffic and reservations

#1site for visitors to NH

Increase staffing and resources for communications

Increased exposure of “visit NH” message

Increased perception of NH as an extreme destination

% increase in agreement with the statement

Website is preferred site for information

Add content, features to web site, keep up to date

% increase in traffic

% agreeing with the statement

# 1 rankings, and time spent on site

The 7 steps to Social Media ROI

1. Define the “R” – Define the expected results?

2. Define the “I” -- What’s the investment?

3. Understand your audiences and what motivates them

4. Define the metrics (what you want to become)

5. Determine what you are benchmarking against

6. Pick a tool and undertake research7. Analyze results and glean insight,

take action, measure again

Step 1: Define the “R” what return do you expect?

If you are celebrating complete 100% success a year from now, what is different about the organization?If you eliminated your department

or failed utterly, what would be different? Typical Outcomes:

Step 2: Define your investment

You can’t divide by $0People timeOpportunity CostExecutive time/goodwill

Step 3: Understand what motivates your stakeholders to engage

What motivates customers to purchase, members to join, students to apply, etc. How does communications

contribute to that engagement?

Step 4: Define your KPIs

Share of thought leadership over timeCost per message communicated

efficiency/efficiency of different channels Increase in employee engagement

in/credibility of communications Improvement in relationship

/reputation scores with customers and communities

Improvement in Optimal Content Score (OCS) over time

Why an Optimal Content Score?

You decide what’s important:Benchmark against peers and/or

competitorsTrack activities against OCS over

time

Optimal content score for social media

Positive: Mentions of the brandKey messagesPositioningVisibility

Negative OmittedNegative toneNo key message

Standard classifications of discussion• Acknowledging receipt of information• Advertising something• Answering a question• Asking a question• Augmenting a previous post• Calling for action• Disclosing personal information• Distributing media• Expressing agreement• Expressing criticism• Expressing support• Expressing surprise

• Giving a heads-up• Responding to criticism• Giving a shout-out• Making a joke• Making a suggestion• Making an observation• Offering a greeting• Offering an opinion• Putting out a wanted ad• Rallying support• Recruiting people• Showing dismay• Soliciting comments• Soliciting help• Starting a poll

Standard classifications of videos

AdvertisementAnimationDemonstrationEvent/

PerformanceFictionFilmHome VideoInstructional VideoInterviewLecture

MontageMusic VideoNews BroadcastPromotional VideoSightseeing/TourSlideshowSpeechTelevision ShowVideo Log

Most frequently used metricsFor External blogs and other Consumer

Generated Media (CGM) Share of positioningShare of preferred types of conversationsShare of desirable vs. undesirableShare of visibilityShare of quotesOptimal content score

For your controlled social media program

TrafficRepeat visits Engagement

Step 5: Define your benchmarks

Past PerformancePeer companiesWhatever keeps the C-suite up

at nightThink 3-5:

A stretch goalThe underdog who’s nipping at your heelsPeer organizationsAnyone that you compete with for share of mind or share of wallet

Emerging benchmarksEngaged = 13 comments per postHyper-engaged = 35 comments per postAfter 3 days most comments are done, 14

days maxSocial Bookmarking momentum = 1

submitted item every other dayMessage should be communicated in 2 out

of 5 blogsPast PerformanceThink 3

PeerUnderdog nipping at your heelsStretch goal

Whatever keeps the C-suite up at night

Possible your benchmarks

Past performance: tonality of blog content

Tonality of Coverage Over Time

4 9 5 9

27

37 43

91

17

914

12

0

25

50

75

100

125

150

Oct Nov Dec Jan

2006 2007

Men

tions

PositiveNeutralNegative

Tonality of all blog postings

Total, 10%

Total, 71%

Total, 19%

The competitive landscape Technorati mentions with high authority

Cingular7%

Sprint7%

Verizon10%

T-Mobile75%

US Cellular1%

Company "sucks" mentions in Technorati with high authority

US Cellular2% Cingular

16%

Sprint12%

Verizon19%

T-Mobile51%

Step 6: Conduct research (if necessary)

First: find out what already exists

Web trafficCustomer Satisfaction dataCustomer Loyalty data

Second: Decide what research is needed to give you the information you need.

Selecting a measurement toolObjective KPI Tool Increase inquiries, web traffic, recruitment

% increase in traffic#s of clickthrus or downloads

Clicktracks, Web trends

Increase awareness/preference

% of audience preferring your brand to the competition

SurveyMonkey, Zoomerang

Engage marketplace Conversation index greater than .8Rankings

TypePad, Technorati

Communicate messages

% of articles containing key messagesTotal opportunities to see key messagesCost per opportunity to see key messages

Media content analysis –Dashboards

% aware of or believing in key message

Survey

The tools you need 1.A content source:

Google News/Google blogsTechnorati, Ice RocketCyberalert, CustomScoop, e-WatchBuzzLogic , Radian 6RSS feedsTweetscan, TwemesSiteVolume

Your tool box also needs to include: 2. A way to analyze that content

Automated vs. ManualThe 80/20 rule – Measure

what matters because 20% of the content influences 80% of the decisions

Dashboards aggregate data

Don’t forget to measure comments as well as YouTube, Facebook & MySpace content

The Optimal Content Score

Your tool box also needs to include: 3. A way to measure

engagementThe conversation index= Ratio of posts to commentsRelationship studiesThe engagement indexTealium

A Proposed Engagement Index

ClickthruDonations/orders

Signups

Time on siteRepeat visits Forwards/links

/comments

RelationshipsTone/content of

conversationMembership

An engagement index?

Output Outtake Outcome

+ +

11 numbers your web analytics guru should give you every month

% increase or decrease in unique visits Change in page rank - i.e. a list of the top ten most popular areas and how it has changed in the last week How many sessions on our blog or web site  represent more than 5 page views In the past  month,  what % of all sessions represent more than 5 page views % of sessions that are greater than 5 minutes in duration % of visitors that come back for more than 5 sessions % of sessions that arrive at your site from a Google search, or a direct link from your web site or other site that is related to your brand % of visitors that become a subscriber % of visitors that download something from the site % of visitors that provide an email address

Courtesy of Eric Peterson

Components of a Relationship IndexControl mutuality

In dealing with people like me, this organization has a tendency to throw its weight around. (Reversed)This organization really listens to what people like me have to

say.Trust

This organization can be relied on to keep its promises.This organization has the ability to accomplish what it says it

will do.Satisfaction

Generally speaking, I am pleased with the relationship this organization has established with people like me.Most people enjoy dealing with this organization.

CommitmentThere is a long-lasting bond between this organization and

people like me.Compared to other organizations, I value my relationship with

this organization moreExchange relationship

Even though people like me have had a relationship with this organization for a long time; it still expects something in return whenever it offers us a favor.This organization will compromise with people like me when it

knows that it will gain something.This organization takes care of people who are likely to reward

the organization.Communal relationship

This organization is very concerned about the welfare of people like me.I think that this organization succeeds by stepping on other

people. (Reversed)

How to implement relationship metrics

Step 1: Conduct a benchmark relationship studyStep 2: Implement PR programStep 3: Conduct a follow up

relationship studyStep 4: Look at what’s changed

Research without insight is just trivia

What works, what doesn’tWhat needs to be done? What are you communicating?What tools work best?

Step 7: Analysis

Data mining the numbers you have

Look for failures firstCheck to see what the competition

is doing Then look for exceptional successCompare to last month, last

quarter, last yearFigure out what worked and what

didn’t work

Case Study: ASPCA

The 2007 Pet Food RecallASPCA ToolbarASPCA Community

(blog, MySpace)Pet Food Recall

Resource CenterUpdates on home pageTo members/

constituents via weekly News Alert

Courtesy: ASPCA

Used Both “old” and “new” Media

Micro-site featuring a “pledge to fight animal cruelty” Dedicated URL:

www.fightcruelty.org Postings on ASPCA’s online

community, MySpace and Facebook pages A radio media tour featuring Ed

Sayres on December 12 Pre- and post-surveys to gauge

post-launch and campaign awareness

Crunching the Numbers

Expense budget: $0CPI (cost per impression): $0.01Every new user is worth $3.50 each to the

organization after 12 months; $6 each after 24 monthsOverall Web traffic jumped, on an annual

average increase, from 25 percent to 52 percentFor every of 100,000 visitors in Web traffic:

Secure 7,000 new registered users Ultimately worth about $175,000 total net to the

organization over their lifetime

Courtesy: ASPCA

Results?

Secured earned media coverage in all top 50 DMAs (goal):

Print: 2.3 million+ impressionsRadio: 7.5 million listenersTV: 16,000,000+ viewers

Raised Web traffic by >22% over November (more than double goal of 10%)Secured 4,700 new registered users via

signing of Anti-Cruelty pledge: 56% > goal (3,000)Pre- to post-survey awareness 28%Expense budget: $130,000CPI: $.10Courtesy: ASPCA

Measuring Exposure

Share of ASPCA Subject Exposure Over Time

3959

5979

100255

32062

117

5638

75

47

135

-

100

200

300

400

500

600

700

800

Qtr1 Qtr2 Qtr3 Qtr4

2007

Mill

ions

Opp

ortu

nitie

s to

see

Adoption / NYC Services Animal Behavior Animal ShelteringAnimal Welfare Tips / Seasonal Tips Anti-Cruelty ASPCA Mission: Orange/No-Kill CommunitiesASPCA Poison Control Center / Toxicology Corporate Humane Law EnforcementEvents

Courtesy: KD Paine & Partners

Measuring ExposureShare of ASPCA Favorable positioning Exposure Over Time

13 2712

28

129

154540

100

25

41

34 221

38

22

10954

165

32

4316 30

-

50

100

150

200

250

300

350

400

Qtr1 Qtr2 Qtr3 Qtr4

2007

Mill

ions

OTS

Expert on pet care

Expert on animal cruelty issues

Expert in veterinary care

Expert in disaster relief/preparedness

Expert in animal welfare issues

Expert in animal toxicology

Expert in animal sheltering issues

Expert in animal behavior

National leader in creating a country ofhumane communities / no-kill

Courtesy: KD Paine & Partners

Social media growth

33,436

37,107

55,097

6,995

27,497

32,576

55,326

5,900

24,145

22,148

41,846

4,500

Blog: Visitors

Online Community: Unique Visitors

Online Community: Total Visitors

Online Community: Members

Blog and ASPCA Online Community GrowthFeb-08 Mar-08 Apr-08

Courtesy: ASPCA

Are they engaged?

Social Media Engagement

36,700

197,800

6,000

4,785

168

13,600

39,918

33,900

178,079

2,900

4,000

140

10,000

21,930

30,000

166,000

1,000

3,000

101

9,000

20,000

MySpace Friends

MySpace ProfileViews*

Facebook Fans

Facebook CauseMembers

Bebo Friends

YouTube ChannelViews *

Flickr Photo views *

Apr-08 Mar-08 Feb-08

Courtesy: ASPCA

KPIs Business Objectives

0

50,000,000

100,000,000

150,000,000

200,000,000

250,000,000

300,000,000

350,000,000

Exp

osur

e

$0

$200,000

$400,000

$600,000

$800,000

$1,000,000

$1,200,000

$1,400,000

$1,600,000

$1,800,000

Don

atio

ns

Overall exposure

Online donations

Georgia Tech Engagement Study

Content from Sep. 1 – Nov. 30, 2007Metrics recorded after 7–30 days,

depending on channel†Athletic content only included on

FacebookCourse blogs not includedPublicly available content onlyOn Facebook, no profiles or names

were recorded

User Generated Media 44

Types and Quantity of Media

Bookmarking Sites (341 items) del.icio.us, Digg, Fark, Newsvine, Reddit, Slashdot

External Blogs (332 posts)50 blogs in 7 categoriesFacebook (811+ discussion

items)Networks and Freshman GroupsInstitution Blogs (1,901 posts,

3,911 comments)114 blogsYouTube (1,668 videos)

30%

10%

26%

25%

19%

12%

11%

11%

11%

5%

12%

55%

31%

26%

29%

22%

24%

33%

8%

YouTube (N=1,705)

Social Bookmarking(N=249)

External Blogs(N=328)

Traditional Media(N=2,748)

Berkeley Carnegie Mellon Georgia Tech MIT Stanford

Share of Desirable Discussion on Uncontrolled Channels

45

Leadership in UGM channels not predicted by traditional media

Georgia Tech was consistently ranked fifth.

Dynamic, channel-specific environment. Most channels varied significantly.

External blogs were the only outlets to mirror traditional media in discussion of orgs.

Social bookmarking favored tech institutes; significantly more likely to mention MIT than any other channel.

User Generated Media 46

During a crisis, UGM channels more likely to be negative

UGM amplified negative traditional media coverage.

Unusual negative stories, like MIT’s fake bomb scare, became popular on social bookmarking sites.

Negative news linked to politics was a mainstay on external blogs.

Facebook profiles amplified each of these effects, and also included critical pieces from campus newspapers.

0.0%0.0%0.0%3.8%

1.9%

14.5%

10.6%

1.9% 0.5%1.3%

13.2%

0.8%

YouTube(1,718 | 194)

SocialBookmarking

(310 | 5)

InstitutionBlogs

(317 | 12)

FacebookPopularTopics

(76 | 21)

External Blogs

(367 | 26)

TraditionalMedia

(2,802 | 154)

Georgia Tech Only All Institutions

Percent of Content Considered Negative Per Channel

47

UGM channels offered equal opportunity for message communication

No UGM channel differed significantly from traditional media in message communication rates.

Among UGM channels, only the difference between external blogs and YouTube can be considered significantly different.

Due to small base sizes, these findings should be considered directional only.

18%

18%

42%

24%

42%

40%

YouTube (N=194)

Social Bookmarking(N=5)†

Institution Blogs(N=12)†

Facebook PopularTopics (n=22)†

External Blogs(N=26)†

Traditional Media(N=154)

Percent of Georgia Tech Content That Communicated One or More Strategic Messages

48

Users were engaged with Georgia Tech content when they found itGeorgia Tech’s most engaging items included a blog post about

its blowable user interface, its Grand Text Auto blog, and a video of a band member “crunking.”

External Blogs

Facebook Discussion Institution Blogs Social Bookmarks YouTube

Avg. Comments

Thread Length

Avg. Comments

Avg. InLinks

Avg. Bookmarks

Avg. Comments

Avg. Comments

Georgia Tech 18 3 2 8,246 659 105 2

Peer 1 11 3 0 590 140 22 7

Peer 2 6 2 1 762 305 40 8

Peer 3 13 5 5 1,405 206 70 4

Peer 4 15 2 1 661 169 36 4

OVERALL 13 3 2 1,687 228 59 5

Thank You!

For more information on measurement, read my blog: http://kdpaine.blogs.com or subscribe to The Measurement Standard:

www.themeasurementstandard.comFor a copy of this presentation

go to: http://www.kdpaine.comOr call me at 1-603-868-1550