the art and science of quality score -...

The Art and Science

of Quality ScoreLearn to improve performance

in the Bing Ads auction

Christi Olson

Bing Ads Evangelist

Bing Ads, Microsoft

@ChristiJOlson

MJ DePalma

Sr. Global Channel & Ecosystem Marketer

Bing Ads, Microsoft

@mjdepalma

John Lee

Bing Ads Trainer

Bing Ads, Microsoft

@john_a_lee

Today’s Webcast Speakers

#BingAdsWebcast

Melissa Mackey

Search Supervisor

gyro

@Mel66

#bingadswebcast

Today’s Guest Speaker

Agenda

• The basics of Quality Score

• Differences between Google & Bing

• 7 hacks for Quality Score

• Using Bing Ads Intelligence to improve Quality Score

Quality ScoreSave time.Improve ROI. Grow your Business.

Quality Score is not a KPI. It’s a diagnostic tool.

Quality Score indicates higher (or lower) CPCs

Keyword Relevance Ad Relevance Landing Page Relevance

The components of quality score:

Does my query match my KW?

Does my ad appear?

Is my ad clicked on?

How does my CTR compare to competitors?

How does my historical CTR compare to my competitors?

Expected CTR

Ad relevance

Is my ad

relevant to the

search query?

Can users find what

they are searching for

on the landing page?

Landing page experience

RELEVANT TRANSPARENCY EASE OF NAVIGATION

How Quality Score Works

Ad Rank = Max CPC * QS

Quality Score is imperfect

Ad Rank = Max CPC * QS

Ad Rank = Max CPC * pCTR

Data on searches

• Impressions

• Clicks

• CPC

• CTR

• Position

• Device

• Location

• Seasonality

• Clickback

17

You can actually get a glimpse into this data from Bing Ads

Share of paid clicks by position

Side Note: 5.3% of search came from “sidebar” ads

0

2,000

4,000

6,000

8,000

10,000

12,000

Mainline

01

Mainline

02

Mainline

03

Mainline

04

Sidebar

01

Sidebar

02

Sidebar

03

Sidebar

04

Sidebar

05

Sidebar

06

Sidebar

07

Sidebar

08

Sidebar

09

Sidebar

10

Clicks by Position

Profile of Marketplace: Avg “Expected CTR” by position

0.00%

2.00%

4.00%

6.00%

8.00%

10.00%

12.00%

14.00%

Mainline

01

Mainline

02

Mainline

03

Mainline

04

Sidebar

01

Sidebar

02

Sidebar

03

Sidebar

04

Sidebar

05

Sidebar

06

Sidebar

07

Sidebar

08

Sidebar

09

Sidebar

10

Profile of Marketplace: Avg. “Expected CTR” by position

0.00%

2.00%

4.00%

6.00%

8.00%

10.00%

12.00%

14.00% 1

2

3

Good

Not as Good

Good

Screen size or device type

vs

Mobile may have lower expected CTR

0.00%

5.00%

10.00%

15.00%

20.00%

25.00%

Mainline

01

Mainline

02

Mainline

03

Mainline

04

Sidebar

01

Sidebar

02

Sidebar

03

Sidebar

04

Sidebar

05

Sidebar

06

Sidebar

07

Sidebar

08

Sidebar

09

Sidebar

10

CTR

Seasonality: Denver in July vs. January?

Seasonality may bring increased CTR expectations

0.00%

2.00%

4.00%

6.00%

8.00%

10.00%

12.00%

14.00%January

July

Check which negative keywords

block your bidded terms

Negative Keyword Conflict Reports

Pause or remove poor performing

keywords

Add relevant keywords to your

account using:

• Keyword Planner

• Campaign Planner

• Bing Ads Editor

• Bing Ads Intelligence

Differences between Google & Bing

Where you access Quality Score information

Keyword, Campaign and

Account level

Broken down by component in

UI reports

Keyword Level

Where you access Quality Score information

In the UI through a

Performance Report, you can

track historical QS over time

Through scripts or third party

platforms/tools

Quality Score components:

eCTR

Ad Relevance

Landing Page Experience

eCTR

Ad Relevance

Landing Page Experience

How Quality Score is calculated

eCTR is a predictive model. There’s no way both Google

and Bing’s engineering teams came up with the same

scoring system.

7 Quality Score Hacks

Hack #1Incremental Geographic Bidding

Seasonality: Denver or Miami?

Find High CVR and CTR regions and get more

Hack #2Add RemarketingBrand affinity (visit homepage)

Audience targeting (RLSA) have advantages in Quality Score

Audience targeting (RLSA) have advantages in Quality Score

Use target + bid:

• Customize ad copy

• Brand affinity drives higher

CTR

• Especially competitive

conquesting

Hack #3Use Extensions

44

Silly.

Somehow people forget.

Where people look. Where the clicks go.

With extensions Without extensions

SUPER ADS! More extensions are better.

Seller Rating

Callout

Review

Social Annotation

Sitelinks

Location

A few days after launching Sitelink Extensions for the Xbox account, Performics saw a 60% increase in traffic and a 51% increase in the click-through-rate.- Performics / Xbox joint case study, Jan 2014

More extensions are better

Hack #4Pick your battles (CPCs)with Share of Voice

Quality Score and Impression Share are delightful together

What share of impressions for the total marketplace are you missing out on – and why?

Sharing of voice reporting = Impression Share

Keyword: Flowers

% Lost due to landing relevance .04%

% Lost due to rank 41% Bids + CTR

% Lost due to keyword relevance 8% CTR

% Lost due to budget 8% Budget

% Lost due to bid 6% Bid

IMPRESSION SHARE 36% WHAT YOU NEED TO CHANGE

Landing page and negative kws

Sharing of voice reporting = Impression Share

Increase budget or change settings to accelerated.

1 BUDGET

Bid more. Your bid doesn’t meet the minimum we require.

2 BID

For quality scores of 8, 9 or 10 bid more, especially if ad position is >3.

3 RANK

1. An ad blocked by rank has passed ALL other filters

2. Rank is made up of 2 components:

MAX BID CTRRANK= x

PREDICTED CTR OF QUALITY SCORE

CVR X CPA

Deep dive on rank

3. Is bid or CTR the problem?

4. Rank is made up of 2 components:

MAX BID CTRRANK= x

PREDICTED CTR OF QUALITY SCORE

CVR X CPA

Deep dive on rank (cont.)

Sharing of voice reporting = Impression Share

Increase budget or change settings to accelerated.

1 BUDGET

Bid more. Your bid doesn’t meet the minimum we require.

2 BID

For quality scores of 8, 9 or 10 bid more, especially if ad position is >3.

3 RANK

Hack #5Start with brand and best keywords for New Accounts

Profile of Marketplace: Avg “Expected CTR” by position

0.00%

2.00%

4.00%

6.00%

8.00%

10.00%

12.00%

14.00% 1

2

3

Good

Not as Good – Lower then expected

Good

Let brand and best terms run

Hack #6Delete high impression poor performing keywords

Periodically delete poor performing high impression terms

Hack #7Improve Ads

Improve ads

Test the framing effect to influence clicks and conversions.

Test framing

Title

Description

CTA

URL

Contacted Hit Bumped Collided Smashed

Sp

eed

, in

MP

H

Word choice leads to higher perceived speed

31

34

38

39

41

Word choices lead to higher perceived speed

Improve ads

Test lively verbs to inspire action.

e.g.

Rock these Cowboy Boots at <Storename>

Test lively verbs to inspire action

Apply the five principles

Pain Emotion Fear Ego Contrast

Apply the five principles

70

“Hate when your garbage bag breaks?”Pain

Apply the five principles

Now apply science

®

Probability of conversion

Clarity of the value proposition (why)

Friction ofelements of process

Motivation of the user (when)

Incentive to take action

Anxiety about entering information

C = 4m + 3v + 2( i-f) -2a

Credit: Marketing Experiments

Focus Tactics Messaging

Anxiety

Credibility

No hidden

fees

Time limit

Since 1984, As Seen on TV, NY Times

No Booking Fees, One-time, No Minimum

Ends by Date, Limited Time

Highlight loss

vs. gainMissing Out, Don’t, Why

Focus Tactics Messaging

Value Prop

Quality

Low price

Selection

High Quality, Durable, Best Selling

Low Price Guarantee, Starting at

$10, Bargain, Wholesale Pricing

Over 5,000 items, Huge Selection

Focus Tactics Messaging

Incentive

Discount

Freebie

Free shipping

Up to 50% off, Save Extra 25%

Free Whitepaper, Buy One

Get One

Free Shipping, Free Overnight

Focus Tactics Messaging

Friction

Quick turnaround

Return policy

Ease of use

Ships Within 1 Day,

Inventory Available

Free returns, 30 day

returns, no hassle

Within Minutes, Quick &

Easy, 3 Steps

We can influence our search audience with smart, effective ad copy.

How a keyword research tool can help you improve quality score

Quality score reporting alone doesn’t lend insight

Doesn’t tell you what you need to fixAd group Keyword Quality score

Ad Group A Keyword 1 1Ad Group A Keyword 2 1Ad Group A Keyword 3 4Ad Group A Keyword 4 1Ad Group A Keyword 5 1Ad Group B Keyword 6 2Ad Group B Keyword 7 2Ad Group B Keyword 8 1Ad Group B Keyword 9 4Ad Group C Keyword 10 2Ad Group C Keyword 11 2Ad Group C Keyword 12 1Ad Group D Keyword 13 2Ad Group D Keyword 14 2Ad Group D Keyword 15 1Ad Group D Keyword 16 2Ad Group E Keyword 17 1Ad Group E Keyword 18 2Ad Group E Keyword 19 3Ad Group E Keyword 20 2Ad Group E Keyword 21 1

Bing Ads Intelligence to the rescue!

Download BAI if you haven’t already

https://aka.ms/intelligencetool

Run a keyword report in Bing Ads

Include the following metrics:• Quality Score

• LP User Experience

• Quality Impact (for keywords with

QS of 5 or less)

Ad group Keyword Quality scoreLanding page

user experienceQuality impact

Ad Group A Keyword 1 10 3 0

Ad Group A Keyword 2 10 3 0

Ad Group A Keyword 3 10 3 0

Ad Group A Keyword 4 6 2 0

Ad Group A Keyword 5 10 3 0

Ad Group A Keyword 6 6 2 0

Ad Group A Keyword 7 6 2 0

Ad Group A Keyword 8 10 3 0

Ad Group B Keyword 9 6 3 0

Ad Group B Keyword 10 10 3 0

Ad Group B Keyword 11 6 2 0

Ad Group C Keyword 12 8 3 0

Ad Group C Keyword 13 6 3 0

Ad Group C Keyword 14 6 2 0

Ad Group D Keyword 15 7 3 0

Ad Group D Keyword 16 6 2 0

Ad Group D Keyword 17 6 3 0

Ad Group D Keyword 18 6 2 0

Ad Group E Keyword 19 7 3 0

Ad Group E Keyword 20 8 3 0

Create a pivot table

Remove keywords with quality score of 0

Now you know what to focus on!

Row Labels Average of Quality score Average of Landing page user experience Average of Quality impactAd Group A 8.32 2.79 0.00Ad Group B 6.60 2.40 0.00Ad Group C 8.14 3.00 0.00Ad Group D 6.00 2.17 0.00Ad Group E 8.43 2.86 0.00Ad Group F 9.13 3.00 0.00Ad Group G 8.67 3.00 0.00Ad Group H 7.17 2.50 0.00Ad Group I 8.27 2.73 0.00Ad Group J 10.00 3.00 0.00Ad Group K 8.22 3.00 0.00Ad Group L 7.67 3.00 0.00Ad Group M 8.63 3.00 0.00Ad Group N 8.67 3.00 0.00Ad Group O 7.07 2.70 0.00Ad Group P 7.52 2.88 0.00Ad Group Q 8.50 3.00 0.00Ad Group R 8.00 3.00 0.00Ad Group S 7.80 2.80 0.00

Use BAI to analyze keywords

• Run the keywords in the worst ad groups through BAI

• Use the “Keyword Categories” function

Check category relevance!

Do the categories relate to your product/service, or are they way off?

Keyword Categories

Keyword Category Scoredrywall moisture proof Home_Improvement/Wall_Coverings 1.0000drywall moisture proof Home_Improvement/Home_Painting 0.2617drywall moisture proof Books_&_Literature/Nonfiction 0.2293moisture control drywall Home_Improvement/Wall_Coverings 1.0000moisture control drywall Home_Improvement/Climate_Control 0.2837moisture control drywall Lawn_&_Garden/Gardening_Supplies 0.2681moisture drywall Home_Improvement/Wall_Coverings 1.0000moisture drywall Home_Improvement/Home_Painting 0.2377moisture drywall Books_&_Literature/Nonfiction 0.2181moisture proof wall board Home_Improvement/Retaining_Walls 0.4368moisture proof wall board Home_Improvement/Wall_Coverings 0.3644moisture proof wall board Home_Furnishings/Rugs_&_Tapestries 0.3579moisture resistent drywall Home_Improvement/Wall_Coverings 1.0000moisture resistent drywall Home_Improvement/Home_Painting 0.2377moisture resistent drywall Books_&_Literature/Nonfiction 0.2181

Use BAI to analyze landing pages

• Run the landing pages through BAI

• Use the “Webpage Keywords” function

Use BAI Webpage Keywords as a check

• Do webpage keywords match your keywords?

• Are they tightly themed?

Webpage Keywords

Input URL Keyword Score(client URL) air barrier systems 0.0597(client URL) barrier systems 0.0583(client URL) performance built 0.0543(client URL) games 0.0475(client URL) air barriers 0.0474(client URL) system overview 0.0440(client URL) controls air 0.0432(client URL) superior systems 0.0417(client URL) temperatures 0.0303(client URL) pre construction 0.0287(client URL) built ins 0.0266(client URL) packaging 0.0251

Takeaways

• Download Bing Ads Intelligence

• Run a report for low QS keywords• Include QS, landing page experience, Quality Impact

• Put into a pivot table to find worst offenders

• Use BAI “Keyword Categories” and “Webpage Keywords” functions to gauge keyword relevance

• Optimize!

Overall Takeaways:

• Relevance is key for Quality Score

• Iterate and optimize to improve Quality Score over time

• Think outside of the box:

Questions?

Ask away!

New to Bing Ads?

Set an appointment with a specialist

to set up your account or campaign:

Need Optimization coaching?

Want to improve your Bing Ads campaign

performance? Schedule a free optimization

appointment with a Bing Ads specialist

We can help you adjust campaign

settings, help you select which

features to use and even walk you

through tools and best practices so

you can make the most of your

Bing Ads investment…. Click here to schedule your

new campaign appointment

Call us!1 877 635 3561 US & CA

Mon - Fri: 6 a.m. to 6 p.m. PT

Sat: 8 a.m. to 6 p.m. PT

Click here to schedule your

optimization appointment

Call us!1 877 635 3561 US & CAMon - Fri: 6 a.m. to 6 p.m. PT

Sat: 8 a.m. to 6 p.m. PT

Get coached by a

Bing Ads Specialist

Webcasts to help you learn.

Bingads.com/webcasts

What’s new in Bing Ads?

https://aka.ms/newatbingads

https://bingads.com/feedback

Thank you!

© 2016 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other

product names are or may be registered trademarks and/or trademarks in the U.S.

and/or other countries.

The information herein is for informational purposes only and represents the current

view of Microsoft Corporation as of the date of this presentation.

Because Microsoft must respond to changing market conditions, it should not

be interpreted as a commitment on the part of Microsoft, and Microsoft cannot

guarantee the accuracy of any information provided after the date

of this presentation.

Microsoft MAKES NO WARRANTIES–EXPRESS, IMPLIED, OR STATUTORY–

REGARDING THEI NFORMATION IN THIS PRESENTATION.