barcelona declaration of measurement principles final with results voting. 20.06.10

Barry LeggetterExecutive Director AMEC

Summit Director

Creating the Barcelona Declaration of Measurement Principles

Moderator

David B. RocklandPartner/CEO

Ketchum Pleon

Barcelona Declarationof Measurement Principles

Global AllianceICCO

Institute for Public RelationsPublic Relations Society of America

AMEC U.S. & Agency Leaders Chapter

Why Are We Here? For the Common Good of Our Industry.

• The communications landscape is changing rapidly

• A lack of clear standards and approaches to PR measurement results in the profession not always being taken seriously; it doesn’t count unless you can count it

• AVEs, random use of multipliers, and other silly metrics and practices diminish the integrity of the profession and the market size

•As a profession, we spend more time arguing amongst ourselves rather than building the field

•We are at a moment in time to make difference, and alter the path we are on, taking advantage of the opportunity to “own” the new communications environment

AMEC International Business Monitor 2010 6

New AMEC research shows strong demand for AVEs

Q8 Current trends in the measurement and evaluation market (base: 34)

47% 32%There is continuing demand for AVEs as a

significant metric for measuring PR programme effectiveness

Agree slightly Agree strongly

Chart does not include % saying “neither agree nor disagree”

Proportion disagreeing

6%

AMEC International Business Monitor 2010 7

Increased Interest in International Measurement and Social Media; ad hoc programmes on the decline

Q8 Current trends in the measurement and evaluation market (base: 34)

Chart does not include % saying “neither agree nor disagree”

Proportion disagreeing

6%

47%

3%

Towards an evaluation standard for marketing PR – market mix modelling

Speaker

Richard HoughtonPresident

ICCO on behalf of New York University

Lou CapozziImmediate Past President, ICCO and Professor, New York

University

• Team of graduate students from NYU’s Masters Degree program in public relations and corporate communications

• Supervised by Prof. Capozzi

• Gained a grant from NYU and support form the International Communications Consultants Organisation

• Broad review of the literature

• Goal – seek commonalities

NYU Student Audit

• Large PR firm models

• Country models• UK, Canada and Germany

• PR measurement vendor models

• Advertising models

• Market mix modelling vendors

NYU Student Audit -Areas Covered

• Real need and appetite for a standardise approach

• Enormous breadth and diversity of existing approaches

• Variation is too great to be rationalised into a single model

• All models have a common flaw

NYU Student Audit –Conclusions

• Broadly adopted by large-scale marketers

• Readily available for PR to connect to

• Enables comparisons between all the elements in the marketing mix

• Fits with many existing PR evaluation techniques

Why market mix modelling?

• Focused tightly on marketing PR, especially for large-scale programs

• Vendor neutral

• Strikes an effective balance between cost and credibility

• Presented in terms familiar to marketers

• Practical with low barriers to entry

Model requirements

• Total PR program costs (negotiated rates)

• Markets covered

• Circulation/audience reach (no multipliers)

• Impressions

• Tone/Quality score

• Impact score

Potential Data Elements

• Up front commitment required

• Historical data needed for MMMs

• Weekly tracking generally required

• Larger PR investments required to show impact

• Budgets need to be tied to PR program elements

Challenges

• Meet with thought leaders• AMEC US and agency research leaders• Corporate pioneers

• Meet with MMM vendors, in-house operations

• Propose a standard

• Get AMEC support

• Seek support from major PR associations

• Promote

Next Steps

John PaluszekChair

Global Alliance For Public Relations and Communication Management

• Mission: To raise standards and promote public relations globally

• Mantra: "We can -- and we must -- all learn from each other"

• Members: National PR associations in 70 countries and leading international PR associations:

• Serving 175,000 practitioners and educators.

• Wide range of services: "Landscapes" of PR in 22 countries . Global PR education study

The Global Alliance For Public Relations and Communication Management

Signature Event

The Stockholm Accords

" Stockholm Accords" Measurement Opportunities

• Effectiveness of public relations communications/messages

• Credibility of source

• Credibility of content

• Familiarity of content

• Long term effectiveness of "Stockholm Accords"

• Integration of public relations into university management curriculum

• Appointments of public relations professionals in C-suites

• Improved respect for the profession in business and general media

The Consultancy ViewProgramme Measurement Principles

Richard Houghton

International Communications Consultancy Organisation

• Umbrella organisation for 1500 PR agencies in 28 countries

• Our members are PR trade associations

• Remit:• Standards – CMS in 15 countries• Forums – senior management• Insight – World Report

• Huge variety in the maturity of individual markets

•

ICCO Members’ Views

The Consultancy View

Principles need to be…

• Based on straight forward and easily understood methodology

• Needs to be understood and used by all account handlers at all levels

• Applicable across PR disciplines

• Structured to allow for growing digital and social media communications

The Consultancy View

We like to think evaluation differentiates• Many agencies feel that their proprietary

evaluation processes give them a competitive edge

• We may, of course ,be kidding ourselves!

• If clients adopt the principles, consultants will too

The ugly reality

AVEs work for the finance guy (mostly)• No they don’t give provide insight into the

effectiveness of a programme or campaign

• No you can’t really compare advertising with PR

• No they don’t take in account tonality or stance

• No they are no use at all for the vast majority of online activity

• But….• …if finance guys want them consultancies

are likely to provide them

Creating the Declaration of Measurement Principles

• Institute for Public Relations Commission on Public Relations Measurement and EvaluationPauline Draper – Chair, IPR Commission

The Commission

Who are we and what do we do?

The Commission is unique in that it comprises some of the leading thinkers from• academia• corporate/government/not-for-profit• agency• research providerOur purpose• to establish best practice for PR research and measurement• to issue authoritative White Papers• to promote excellence in PR research and measurement through the Jack Felton Golden Ruler Awards and partnering with others

Example White Papers

Available from InstituteforPR.org

•A primer on measuring corporate reputation: a case study on the Dow Chemical Company

•Doing Measurement Right: One Organization’s Experience Creating a Best-In-Class Measurement Program from Scratch

•Isolating the Effects of Media-based Public Relations on Sales: Public Relations Optimization Through Marketing Mix Modeling

•Using Web Analytics to Measure the Impact of Earned Online Media on Business Outcomes: A Methodological Approach 

•A New Paradigm for Media Analysis: Weighted Media Cost

•Guidelines for Setting Measurable Public Relations Objectives: An Update

Definition: The calculation of space or time used for earned media (publicity or news content) by comparing it to the cost of that same space or time if purchased for advertising. AVE is used because it assigns a monetary value to public relations using a metric that is simple to understand, simple to calculate, and uses data that is readily available.

•AVE suggests that the space/time occupied by earned media is equivalent to the same space/time purchased•AVE is not a proxy for measuring public relations’ ROI

Advertising Value Equivalents

Why not?

Outcomes rather than Outputs or Outtakes

•Measurement and evaluation practices should demonstrate the effectiveness of public relations to help meet organizational goals.•Measuring the quality of public relations output is necessary in order to demonstrate its contribution to reaching these goals.•Any measure of media coverage should include variables such as tone, prominence, placement, appearance of key messages, the portion of story that applies to the organization or its key messages, and the credibility and targeted reach of the medium in which the message appears.•Whenever possible, it is best to link media coverage with expected outcomes. These outcomes should also be tied to organizational goals.

Advertising Value EquivalentsMeasuring the Value of this Marketing

Communication

Measuring OUTCOMES

• Benchmarking:– To past performance– To closest rival– To competitors

• Map outputs to other metrics:– Share price– Sales– Web analytics data

• Specific research:– Pre/post survey– Message testing– Journalist audit

• Utilize other data:– Surveys– Focus groups– Marketing mixed modeling /

econometric modeling– Client and Employee

satisfaction studies

Practices for taking measurement further

Barcelona Declarationof Measurement Principles

Public Relations Society of AmericaGary McCormick

Chairman and CEO

As a first step, ensuring that four basic questions are asked regarding the work that is to be undertaken:

•Whom are you seeking to affect?•What about them are you seeking to affect?•How much must they be affected to be successful?•By when does this effect need to occur?

The Business Case for Public Relations

A framework for building measurement into public relations programs – a

lexicon and recommended approaches.

Financial•Generates Revenue, Sales, Profit; Marketing public relations drives sales.•Investor public relations drives investment, valuation.•Public relations drives donations and membership for relevant organizations.Reputation / Brand Equity•Increases likelihood to purchase / consider your brand(s).•Minimizes the effects of a crisis and rebuilds trust. Establishes credibilityEmployees and other Internal Publics•Increases employee satisfaction and engagement, leading to greater efficiency, increased retention, reduced turnover, lower recruiting costs and higher productivity.Public PolicyCreates public awareness, understanding and support for legislation, regulation and political candidates.Affects voter behavior. Helps pass legislation, regulation and initiatives.

As a second step, connect the desired outcomes to strategic goals – one of four

key areas:

Financial•Consumer response: Field consumer survey; determine purchase levels and exposure to public relations results; isolate causal effects through statistical analysis. •Market Mix Modeling / Econometric Modeling: Gather public relations output / outcome data by function, market, region and time period; factor revenue-generation by market, by region and over time. Apply regression analysis.Reputation / Brand Equity•Benchmark reputation / relationship metrics via survey prior to a campaign; repeat every three to six months.•Correlate attitudinal studies with customer purchase attitudes and behavior.Employees and other Internal Publics•Compare control groups to employee populations exposed to public relations activities.•Focus on performance outcomes, not attitudes or awareness, such as employee satisfaction and engagement findings, employee turnover statistics and other recruitment data.Public PolicyUse tracking services to track awareness, correlate to public relations activity, and connect to actual votes recorded and legislative outcomes.

Third, measure the desired outcomes not in terms of outputs – clips or activity – but in terms

of outcomes:

Barcelona Declarationof Measurement Principles

AMEC U.S. & Agency Leaders Chapter

Tim MarkleinExecutive VP, Measurement & Strategy

Weber Shandwick

AMEC’s first international chapter. Launched in fall 2009. Provides a networking forum for U.S. research and agency leaders. Represented on AMEC Board by inaugural chair David Rockland of Ketchum. Includes 16 members:

Existing AMEC Members:BurrellesLuce, Cision, Dow Jones, Echo Research, Report International, VMS

Large Agency Research Leaders:Edelman/StrategyOne, Fleishman-Hillard, Hill & Knowlton, Ketchum, MS&L, Ogilvy, Weber Shandwick

Midsize Agencies and Specialist Research Firms:Chandler Chicco, Evolve24, Leading Communicators

U.S. & Agency Leaders Chapter

Topic #1. Ad Value Equivalency

Building on IPR Commission’s Oct’09 vote to “condemn the name, concept and practice of AVEs,” we agreed to the following fundamental principles:

1.This topic is germane to AMEC.2.Outcome based metrics – attributes, behavior, business results, etc. – should always be used in preference to output metrics, whenever possible.3.When measures are based on cost of media space or time, they should be referred to as “costs” not “values.” 4.If media costs are used, they should be adjusted for quality (tone, message, etc.) and for the actual space or time occupied by a client or competitors.5.Multipliers should not be used unless validated by third-party audits, surveys or market mix models.6.Negotiated rates should be used (not rate cards).7.Methodologies must be transparent and repeatable.

Topic #2. Social Media Measurement(part one of two)

Building on AMEC U.S. member experience, we agreed to the following fundamental principles:

1.Social media can be measured.2.Social media measurement is a discipline and a process, not a tool.3.There is no “single metric” for social media.4.Media content analysis has a fundamental role in social media measurement – and should be supplemented by web and search analytics, sales and CRM data, survey data and other methods.5.Organizations need clearly defined goals and outcomes for social media in order to measure it.

• Refer to PRSA “Business Outcomes” outline:http://comprehension.prsa.org/?p=628

• Refer to Altimeter Group “Social Marketing Analytics” outline: http://bit.ly/dldIHf

Topic #2. Social Media Measurement(part two of two)

Building on AMEC U.S. member experience, we agreed to the following fundamental principles:

6.Evaluating quality and quantity together is critical, just as it is with conventional media.7.Given the scale and volume of social media, technology-assisted analysis is necessary – but human reading and coding is still valuable for precision analysis of tone, messaging, nuance, etc.8.Measurement must adapt to the media, focusing on “conversation” and “communities” not “coverage.”9.Understanding reach and influence is important, but existing sources are not accessible, transparent or consistent enough to be reliable.10.Experimentation and testing are key to success – though measurement and communication fundamentals still apply.

Topic #3. Market Mix Modelling

Building on the initial NYU market mix modeling report and AMEC U.S. member experience, we agreed to the following fundamental principles:

1.Clients are creating demand for market mix models to evaluate the impact of consumer marketing.2.The PR industry needs to understand the value and implications of market mix models for accurate evaluation of consumer marketing PR, in contrast to other measurement approaches.3.The PR industry needs to develop PR measures that can provide reliable input into market mix models.

• Such measures might focus on media quantity and quality, messaging and audience impact, traditional versus social media, consistency with other marketing disciplines, etc.

Topic #4. TransparencyClients are entitled to know how their content was obtained and the methodology used to append any metadata to that content:

•Sources:Print - Physical clippings, aggregator feeds, website linksBroadcast – closed caption text, speech to text, manual observationInternet – aggregators, spiders, text scrapes, screen viewsBlogs – main post only or post plus comments and forwards

•Qualitative assessments:Human or machine scored, sampled or all coverageTone – criteria for scoring should be clear and sharedReach metrics – the source of the data and the criteria they are based upon should be clear, i.e. persons 2+, persons 18+, daily or monthly visitors, unique visitors, etc.Other – content analysis of messages, scorecards, calculated measures such as impact scores or value scores should be understandable and not a “black box”

AMEC International Business Monitor 2010 46

Networking Refreshment Break

A Declaration of Measurement Principles

David Rockland Partner / CEO, Managing Director

KetchumAMEC Board Member

IPR Measurement Commission, Past Chairman

TELE-VOTING SYSTEM

The debate: how to vote

1.You will be advised when to vote2.You will be given clear options on which to vote3.Please click your preferred option during the voting time4.To change your option, click again. The last one you record will be the one accepted by the system5.The result will be shown on the screen once all voting finishes6.Summit staff will collect the voting keypads at the end of the conference7.Further opportunity for comment to david.rockland@ketchum.com until July 15

Our Summit Voting Test!

Q. Is Catalan the officially spoken language in Barcelona?

• YES: Press 1

• NO: Press 2

Seven Principles

1. Importance of Goal Setting and Measurement 2. Media Measurement Requires Quantity and Quality3. AVEs are not the Value of Public Relations 4. Social Media Can and Should be Measured 5. Measuring Outcomes is Preferred to Measuring

Media Results 6. Business Results Can and Should Be Measured

Where Possible 7. Transparency and Replicability are Paramount to

Sound Measurement

Principle 1:Importance of Goal Setting and Measurement

• Goal-setting and measurement are fundamental aspects of any public relations program

• Goals should be as quantitative as possible and address who, what, when and how much the PR program is intended to affect

• Measurement should include representative traditional and social media; target audience changes in awareness, comprehension, attitude, and behavior as applicable; and business results

The Seven Principles

Principle 1Importance of Goal Settingand Measurement

• YES: Press 1

• NO: Press 2

Principle 2: Media Measurement Requires Quantity and Quality

Overall clip counts and general impressions are usually meaningless. Instead, media measurement, whether in traditional or online channels, should account for:

•Impressions among the target audience

•Quality of the media coverage including:ToneCredibilityMessage DeliveryInclusion of a 3rd party or company spokespersonProminenceVisual dimensions

•Quality can be negative, positive, or neutral

The Seven Principles

Principle 2 Media Measurement Requires

Quantity and Quality

• YES: Press 1

• NO: Press 2

Principle 3: AVEs are Not the Value of Public Relations

• Advertising Value Equivalents (AVEs) do not measure the value of public relations and do not inform future activity; they measure the cost of media space

• Where a comparison is to be made between the cost of space from earned versus paid media, validated metrics such as Weighted Media Cost should be used and reflect:

• negotiated advertising rates• quality of the coverage (see Principle 2), including negative results• physical space of the coverage, and the portion of the coverage that is relevant

• Multipliers intended to reflect a greater WMC for earned versus paid media should never be applied unless proven to exist in the specific case

The Seven Principles

Principle 3

AVEs are Not the Value ofPublic Relations

• YES: Press 1

• NO: Press 2

Principle 4: Social Media Can and Should Be Measured

• Social media measurement is a discipline, not a tool; but there is no “single metric”

• Organizations need clearly defined goals and outcomes for social media

• Media content analysis should be supplemented by web and search analytics, sales and CRM data, survey data and other methods

• Evaluating quality and quantity is critical, just as it is with conventional media

• Given the scale and volume of social media, technology-assisted analysis is necessary – human reading and coding is valuable for precision analysis of tone, messaging, nuance, etc.

• Measurement must focus on “conversation” and “communities” not “coverage”

• Understanding reach and influence is important, but existing sources are not accessible, transparent or consistent enough to be reliable; experimentation and testing are key to success

The Seven Principles

Principle 4 Social Media Can and Should

Be Measured

• YES: Press 1

• NO: Press 2

Principle 5: Measuring Outcomes is Preferred to Measuring Media Results

• Outcomes include shifts in awareness, comprehension, attitude and behavior related to purchase, donations, brand equity, corporate reputation, employee engagement, public policy, investment decisions, and other shifts in target audiences regarding a company, NGO, government or entity, as well as the audience’s own beliefs and behaviors

• Benchmark and tracking survey research are the preferred practices for quantitative measurement

• Standard best practices in survey research including sample design, question wording and order, and statistical analysis should be applied in total transparency

The Seven Principles

Principle 5

Measuring Outcomes isPreferred to Measuring Media Results

• YES: Press 1

• NO: Press 2

Principle 6: Business Results Can and Should Be Measured Where Possible

• Models that determine the effects of the quantity and quality of PR outputs on sales or other business metrics, while accounting for other variables that drive sales, are a preferred choice for measuring consumer or brand marketing. Related points are:

Clients are creating demand for market mix models to evaluate

the impact of consumer marketing The PR industry needs to understand the value and implications

of market mix models for accurate evaluation of consumer marketing PR, in contrast to other measurement approaches

The PR industry needs to develop PR measures that can provide reliable input into market mix models

Survey research can also be used to isolate the change in purchasing, purchase preference or attitude shift resulting from exposure to PR initiatives

The Seven Principles

Principle 6

Business Results Can andShould Be Measured Where Possible

• YES: Press 1

• NO: Press 2

Principle 7: Transparency and Replicability are Paramount to Sound

MeasurementPR measurement should be done in a manner that is transparent and replicable meaning the detailing of:

Media Measurement:• Quantitative – source of the content: print, broadcast, internet, consumer generated media along with criteria used for collection• Qualitative – human or automated, tone, reach to target, content, overall media impact

Surveys:• Methodology –sampling frame and size, margin of error, probability or non-probability• Questions – all should be released as asked• Statistical methodology- how specific metrics are calculated

The Seven Principles

Principle 7

Transparency and Replicability are

Paramount to Sound Measurement

• YES: Press 1

• NO: Press 2

Next Steps:

Panel Organizations

Delegates

AMEC

THE RESULTS Yes No1. Importance of Goal Setting and Measurement 97% 3%

2. Media Measurement Requires Quantity and Quality 97% 3%

3. AVEs are not the Value of Public Relations• Do not measure the value of PR or future activity• Where comparisons made validated metrics should be used• Multipliers never be applied unless proven to exist

70%92%57%72%

30%8%43%28%

4. Social Media Can and Should be Measured 93% 7%

5. Measuring Outcomes is Preferred to Measuring Media Results 87% 13%

6. Business Results Can and Should Be Measured Where Possible 86% 14%

7. Transparency and Replicability are Paramount to Sound Measurement

95% 5%