display advertising’s impact on online branded...

Display Advertising’s Impact on Online Branded Search

Randall Lewis∗ Dan Nguyen†

Yahoo! Research University of Chicago

June 25, 2012

Abstract

We study how display advertising influences customers’ online searches for the ad-vertiser and its competitors and how these influences might affect the search advertis-ing market. We show that display ads increase online searches for both the advertisedbrand and its competitors’ brands. We did this using data from a natural experi-ment that randomized the delivery of display ads to visitors to Yahoo!’s front page(www.yahoo.com). From studying three advertising campaigns, we found that displayads increased searches for the advertised brand by 30% to 45% while increasing searchesfor its competitors’ brands by up to 23%. Furthermore, the total number of incrementalsearches for the competitors’ brands was 2 to 8 times more than that for the adver-tised brand. We also explore the number of exposures as a moderator to these effectsand found evidence suggestive of wearing-in and mild wearing-out of the effect. Wetranslated this search lift to a search revenue lift for the publisher who owns the searchengine and found that one full day of display advertising on Yahoo!’s front page gener-ates 100 USD to 1, 000 USD of search revenue, just from the brand-specific keywordswe considered. In addition, our empirical results imply that online display advertisingcan influence the bidding behavior of advertisers in search advertising auctions.

Keywords: search advertising, display advertising, advertising synergies, multimedia ad-vertising effectiveness, strategic complements of advertising, natural experiments

∗This work was completed by the authors while at Yahoo! Research. Randall Lewis is now employed byGoogle, Inc. and can be contacted at [email protected], 1600 Amphitheatre Parkway, Mountain View,CA 94043.†Dan Nguyen can be contacted at [email protected].

1

1 Introduction

Online advertising has introduced a number of new advertising channels to the traditional

offline channels such as print media and television. These new channels include search,

display, online classified ads, lead generation, online video, video games, and mobile phones.

The market for these channels has been growing throughout the past decade as indicated by

the growing share of total media expenditure on online advertising.1 Because so much money

is invested in these channels, it is important for both advertisers and online publishers to

understand how these new channels interact with each other.

This paper specifically studies the interactions between the two largest online advertising

channels: search and display advertising. Recent investment trends in search and display

advertising might suggest that the channels are substitutes for each other. Search ads grew

from 1% of online advertising revenue in 2000 to 44% in 2008 while display ads dropped from

40% of online advertising revenue to 21% of total revenue during the same period (Evans

2009). The drop in display advertising revenue share is mostly due to advertisers’ belief

that display advertising is a less effective advertising channel than search advertising. In a

2009 Forrester research survey, 9% of retailers rated banner ads as effective (Mintel 2010).

However, these advertisers might not realize the indirect effects of display advertising on

online search behavior. We hypothesize that display advertising causes people to search

about the advertised product because of priming. An advertiser’s display ad should also

cause people to search for the competitors’ brands because it not only primes the advertised

brand but also product category.

To test these hypotheses, we exploit exogenous variation generated by a natural exper-

iment on Yahoo!’s front page (www.yahoo.com).2 On certain days, Yahoo! sells the main

display ad space on the front page as an “ad split” in which two display ads alternate every

1From 2009 to 2010, US online advertising expenditure grew from 22.8 to 26.1 billion USD, a 14.5%increase, while traditional media grew from 124.4 to 126.9 billion USD, a 2.0% increase (Mintel 2010).

2Lewis (2012) uses this same variation to examine the impact of frequency on clicks and new accountsign-ups.

2

second throughout the day. As long as visiting the front page on an even or odd second is

orthogonal to a visitor’s response to the display ad, the ad split creates exogenous variation

to identify the causal effects of the display ad. To take advantage of this natural experiment,

we recorded front page visitors’ ad exposures and subsequent search behaviors on three ad

split days when one of the advertisers was from a well-defined product category (the target

ad) and the other advertiser was from an unrelated category (the control ad). We compared

the impact of exposure to the target and control ads by examining users’ search behavior

immediately following visits to the front page. Focusing on the ten minutes following expo-

sure, we find that the display ads caused a 30 to 40% increase in searches for the advertised

brand and a 1 to 6% increase in searches for any competitors’ brands. When analyzing the

display ad’s effect on searches by brands, we found as high as a 23% increase in searches for

a competitor’s brand. There was significant heterogeneity in the effect across competitors’

brands, but no competitor experienced a statistically significant decrease in searches from

the display ad. Therefore, display advertising increases the likelihood of consumers to search

for many of the brands in the product category.

After establishing that these effects exist, we investigate how the number of display

exposures moderate them. Lewis (2012) finds that increasing the number of deliveries of a

display ad to a user decreases the likelihood of clicking on the ad for half of a sample of

30 front-page display ad campaigns. From this past finding, we expect that increasing the

number of exposures will also decrease the display advertising effects on online searching, but

we find little evidence of this across the three ad campaigns. This is particularly important

to publishers who are using the cost-per-click business model of display advertising because

it determines how they should rotate their display advertisements to a user. Given that

the effects on searching do not diminish with multiple exposures, publishers might want to

encourage multiple exposures of a display advertisement to a user in order to capture extra

search ad revenue sales from the search spillovers even though the propensity to click the

display ad falls.

3

In the last sections, we give some indications of the economic and strategic impact of these

search spillovers. Using back of the envelope calculations, we convert these extra searches

to the extra revenue generated for the publisher and search advertisers. We specifically

calculated the extra revenue from one day of search advertising caused by one day of display

advertising on Yahoo!’s front page. To calculate publisher’s revenue, we took cost-per-click

(CPC) and click-through-rate (CTR) data from Microsoft Ad Center and converted those

extra searches generated by display advertising to revenue. We found that one display

advertisement in one day generated from 96 USD to 1, 032 USD in expected revenue for the

publisher from the branded search keywords that we considered.3

We also argue that our results imply that display ads complement the advertisers and

competitors’ search ads, both directly and indirectly. Display advertising indirectly lowers

the fixed cost or “entry” cost of advertising on that search result page because it increases

the reach of the search ad by increasing the traffic to the search result page. However, display

advertising is a more direct complement, thanks to the generalized second price auction of

the search advertising market. In the search advertising market, advertisers pay for a search

ad on a per click basis, and the CPC is lower for positions with lower click through rates.

4 If the objective for the search advertiser is to get clicks, an advertiser can get the same

number of clicks in a day with a lower expected CTR when there is an expansion in the

number of searches. By bidding for a position with a lower click through rate, an advertiser

lowers its CPC and increases its marginal profitability for a given number of clicks. Similarly,

advertisers can also do this in a response to increases in searches from its competitors’ display

ads. As such, we see both complementarities and, in the language of Bulow, Geanakoplos,

and Klemperer (1985), strategic complementarities of display advertising on search.

These findings will be of interest to advertisers, publishers, and regulators. Search adver-

3For statistical and practical reasons, these numbers do not consider the incremental revenue from allimpacted queries such as generic category keywords or other “long-tail” keywords. For the purpose of thispaper, we have focused on branded category keywords.

4Actual CPC paid by the advertiser is determined through a generalized second price auction. For moreinformation about this auction, see Edelman, Ostrovsky, and Schwarz (2005).

4

tisers should consider adjusting their bids in the search ad auction or at least have search ads

when they or their competitors run display advertisements on heavily trafficked sites such

as Yahoo!. In addition, display advertisers should highlight their differential value know-

ing consumers will search for other products in the category, ensure they have a consistent

marketing message across the two channels, and coordinate with their distribution partners

who advertise on the brand’s keyword search result pages. On the supply side, publishers

who own only one of the two channels should consider developing a business in the other

channel either through investment in research and development, partnership, or acquisition

to capture the full revenue from both channels. In fact, this market organization where

publishers own both channels and compete on systems instead of individual channels is the

welfare-maximizing outcome for the online advertising market because publishers would in-

ternalize these complements when making pricing decisions. This is true for any markets

with complement products as discussed in Economides and Salop (1992).

The paper is organized as follows: Section 2 reviews related literature. Section 3 details

the natural experiment and the data. Section 4 describes the advertising campaigns and

search keywords we used. Section 5 summarizes the data and confirms the randomization.

Section 6 describes the empirical models and summarizes the results. Section 7 explains the

economic impact of our findings. Section 8 discusses limitations and concludes.

2 Related Literature

Our hypothesis comes from research suggesting that ads motivate consumers to learn and,

hence, search for more information about the product category.5 In a theoretical model, May-

zlin and Shin (2011) show that there exists a separating equilibrium where a high quality

firm would choose an advertisement devoid of any attribute information in order to invite

the consumer to search. In laboratory settings, Swasy and Rethans (1986) found that adver-

tising for new products created more curiosity among consumers with high product category

5The information provided by display advertising is limited by the nature of the medium.

5

knowledge. Menon and Soman (2002) show that curiosity-inducing advertising increased

time spent and attention on gathering information but did not increase the number of clicks

on links for more information. In regards to priming other brands in the category, Nedun-

gadi (1990) found that priming of a minor brand in a less accessible subcategory significantly

increases the retrieval and consideration of the major brand in the same category.

The only empirical study that we know of that tries to measure the effects of advertising

on online search behavior using field data is by Joo, Wilbur, and Zhu (2011). They show that

there is a significant correlation between television advertising for financial services brands

and consumers’ tendencies to search online for these brands. We add to this literature

by using the exogenous variation created by a natural experiment to show that display

advertising causes consumers to search online for both the advertised brand and other brands

in the same product category.

There is also limited research on the interactions between online advertising channels.

Most research on the effectiveness of a multimedia advertising strategy has concentrated

on the interactions between offline channels and online and offline channels. Naik and Ra-

man (2003) established that there were synergies between print and TV ads. In the lab,

Chang and Thorson (2004) found that television and web synergies lead to higher attention,

higher perceived message credibility, and more positive thoughts than did repetition within

the same media. Goldfarb and Tucker (2010) showed that prices per click for search adver-

tisements are 5 to 7% higher when lawyers cannot contact potential clients by mail. They

interpreted this result that online advertising channels are substitutes for offline advertising

channels. Our paper fills some of the gaps on this issue by showing and measuring some of

the complementarities between the online display and search advertising channels.

6

3 The Natural Experiment and Data Collection

The data was generated in a natural experiment with display ads on Yahoo!’s front

page (www.yahoo.com). Yahoo! participates as a publisher in both the search and display

advertising markets. In 2010, approximately 34% of Yahoo’s revenue came from display

while 50% come from search.6 The front page is one of its most heavily trafficked sites.

Approximately 40 million unique users visit the front page on any given day. Figure 1 is a

screenshot of the front page and shows a Progressive display ad from our study. We collected

anonymized data on users’ ad exposures on the front page and subsequent search behaviors.

We collected this data on days with ad splits because ad splits produce the exogenous

variation needed to identify the effects of display advertising. On any given day, an advertiser

can purchase a display ad on Yahoo!’s front page for the whole day (i.e., all impressions) or

purchase a spot in an ad split (i.e., half of the day’s impressions). If an advertiser chooses to

be part of a split, its ad is shown to visitors who arrive at Yahoo!’s front page on, for example,

any odd second, while another advertiser’s ad is shown in the same position on the webpage

to those who arrive at the site on any even second. As a result, each user has a 50% chance

of being exposed to either ad in the split during each page visit. In addition, the number of

impressions of a given ad delivered to a user will be distributed binomial(v, .5) where v is

the total number visits to the front page. If there is no systematic difference between odd

and even second visits then the ad shown at each impression and the percentage of the total

number of impressions delivered to a user are exogenous.7 Therefore, front page ad splits

provide a natural experiment with which we can study the effects of advertising.

Each ad impression defines a treatment or control group event. Specifically, we associate

all behavior immediately following the delivery of the target ad with our test group and all

behavior following the delivery of the other split ad with our control (baseline) group. A

period begins upon the delivery of an impression and ends either when another impression

6Yahoo! 10-K FY 20107We refer the interested reader to Lewis (2012) who discusses this experiment and its virtues and failings

in greater depth.

7

is delivered to the same user or after ten minutes have elapsed, whichever comes first.8 Ten

minutes should be long enough to allow for users to perform enough activities after the

delivery of an impression but short enough to avoid misattributing activities to the delivered

impression.9

We recorded each anonymous user’s searches on Yahoo! during each ten-minute period

after an impression was delivered. We focused on searches related to the brands of the

advertised products and their competitors’ brands. A search was defined as related to the

advertiser’s or a competitor’s brand if it included the name of the brand or its product.10

Finally, we removed from our data the 0.02% of users who visited the front page more

than 100 times. Such extreme behavior is more characteristic of webcrawlers (i.e., computer

programs) than humans.11 Less than one percent of users visited the front page more than

30 times.

4 Advertising Campaigns and Search Keywords

This paper studies display advertising campaigns for Progressive Car Insurance, Acura

TSX sports wagon, and Samsung Galaxy Tab. Each of these three campaigns was part of a

front page split with a control display ad unrelated to the advertised product category. The

Progressive campaign was part of a split with an ad for a TV show; the Acura TSX campaign

was part of a split with an ad for an electronic retail store; and the Samsung Galaxy Tab

campaign was part of a split with an ad for an action movie. Table 1 shows the creatives for

each display ad campaign and the corresponding control display ads in their splits.

8Only 18% of impressions were followed by another impression from the study within ten minutes.9Examining behavior during the ten minutes following a user’s exposure to a display ad impression

yielded the most statistically powerful results in independent tests. While we may underestimate the totalnumber of searches caused by the display ads, we intentionally make this tradeoff to focus on the time windowwhen the signal to noise ratio is sufficiently large to make meaningful statistical measurements.

10We acknowledge that this method will incorrectly categorize some searches as searches for the brand’sproduct, but instead, they are really searches for topics related to the brand like queries for financial state-ments for the brand. After scanning some of the categorized searches, we believe that most if not all of thecategorized searches are searches for the brands’ products.

11The maximum number of visits to the front page by a unique user in a single day was around 4, 000,averaging once every 15 seconds during waking hours.

8

To capture as much of the complementarities between display and search advertising, we

wanted to consider all the brands in the advertised product category or market as search

keywords. To identify other category brands for the Progressive Auto Insurance campaign,

we used a December 2009 Mintel Marketing Intelligence research report about the Auto

Insurance Industry.12 In total, we found 14 brands other than Progressive Auto Insurance.13

These competitors include all the top 10 underwriters, which make up over 67% of premiums

written in 2008 (Mintel 2009).

For competitor-branded search keywords for the Acura campaign, we used all makes listed

on Autobytel.com, a consumer website about passenger vehicles. In total, we identified 36

brands other than Acura that range from standard brands such as Ford to sports and luxury

brands such as Porsche.14

Finding the category brands for the Samsung Galaxy Tab campaign was more difficult.

During June 2011, the month of the ad split, the tablet PC market was fairly new and

dominated by one product, Apple’s iPad; however, new products were entering the market

on a monthly basis. To summarize these options and upcoming releases, CNET, a media site

about technology products, wrote an article titled “CNET looks at current and upcoming

tablets.” In this article, they listed all upcoming and currently available tablet PCs from

March to the end of 2011.15 From the article, we used the 15 products and brands listed as

potential search keywords.16

For the three advertising campaigns, we have identified many brands in each advertised

12The report contains the results from a survey asking adults 18 and over whose households own aninsured vehicle about their preferred auto insurance company. We took those that were named as otherbrands in the market.

13The brands other than Progressive are State Farm, Allstate, Geico, Farmer’s Insurance, NationwideInsurance, Liberty Mutual, USAA, AIG, American Family Insurance, 21st Century Insurance, TravelersInsurance, Hartford AARP, Erie Insurance, and Safeco.

14The brands other than Acura are Audi, BMW, Cadillac, Chevrolet, Dodge, Ford, GMC, Honda,Hyundai, Infiniti, Jeep, Kia, Lexus, Lincoln, Mazda, Mercedes, Mini, Mitsubishi, Nissan, Scion, Subaru,Suzuki, Toyota, Volkswagen, Volvo, Buick, Chrysler, Fiat, Jaguar, Land Rover, Porsche, Rolls Royce, Saab,Smart, and Tesla

15Retrieved July 29, 2011.16The brands other than Samsung’s Galaxy Tab are Dell Streak, Motorola Xoom, HTC, Blackberry

Playbook, Asus, Acer Iconia, Apple iPad, Amazon Tablet, Sony, G-Slate, Toshiba, Vizio, Viewsonic, andMicro Cruz.

9

markets, but we might have ignored some close substitutes. For example, the Samsung

Galaxy Tab also competes with e-readers, smartphones, netbooks, and ultra portable laptops.

By ignoring these product market substitutes, our estimates are more conservative and

underestimate the magnitude of the possible search spillovers.

5 Data Summary

Table 2 summarizes the data. There were 38-41 million unique users who visited the front

page 161-171 million times each day of the splits. As expected by the natural randomization

created by the ad split, the target ad was delivered on 50% of those visits while the control

ad was delivered on the other 50% .

A small fraction of users searched for any category brands on the day of the campaign.

Only 0.06% of front page visitors searched for the advertised brands and its competitors on

the day of the Progressive Auto Insurance ad campaign, 0.8% searched on the day of the

Acura TSX ad campaign, and 0.04% searched on the day of the Samsung’s Galaxy Tab ad

campaign. In spite of these small percentages, the scale of the Yahoo! front page still yielded

a large number of users who search: 15, 545, 331, 047, and 24, 258 users, respectively, for the

three campaigns.

Most visitors to Yahoo!’s front page in a day return at least once that same day. Figure

2 shows three histograms of the total number of users visiting for each day of the three

campaigns; each histogram is heavily skewed right with the mean visits in a day exceedingly

the median of two. On average, a user visits the front page four times during the day and,

as a result, sees four front page display ad impressions, randomly split between the target

and control ads.

Figure 3 shows the distribution of the number of target ad exposures delivered to each

user for the three campaigns. This distribution is also heavily skewed right. As a result,

the distribution of number of target ad exposures is driven mostly by the total number of

10

visits and not by the randomization from the ad split. In fact, greater than 80% of the

variation in target ad exposures is attributable to the total number of visits. Given v, the

total number of visits, the number of target ads shown to a user should be distributed

binomial(v, 0.50) where v is the total number of visits. Figure 4 shows four histograms:

three showing the empirical distribution of target ad exposures for users who visited the front

page ten times and, for comparison, one plotting the binomial distribution for n = 10 and

p = 0.50. The similarity of the sample distribution to the theoretical distribution confirms

the impression-level randomization mechanism used by the ad splits. However, knowing that

not all variation in target ad exposures is randomized, we will need to carefully control for

users’ online behaviors that may cause ad exposure and searching intensity to be correlated

due to “activity bias” outlined by Lewis, Rao, and Reiley (2011).

6 Empirical Analysis and Results

In this section, we measure the impact of display advertising on branded search from

the data we collected. In our first analysis, we calculate the difference in the likelihood to

search for a brand between our test and control groups. With this estimate, we can easily

calculate and understand the total impact of the ad by multiplying the estimate by the total

number of impressions. In our second analysis, we measure how the number of ad exposures

moderates the effect. We do not use this analysis to recalculate the total impact because we

would obtain a very similar estimate to our conceptually simpler first analysis.

6.1 The Total Impact of Display Advertising on Search

Our first analysis compares the search probability for a brand between our test and control

groups. To measure the difference in probabilities, we estimated the linear probability model

Searchijt = αj + βjADit + εijt (1)

11

using ordinary least squares (OLS) for each brand j on the data collected from the ad

campaign relating to brand j’s product category. Searchijt is an indicator variable equal

to one if user i during period t searched for brand j and zero otherwise. ADit is another

indicator variable equal to one if the target ad was delivered to user i during period t. εijt

is the error term.

The estimated βj unbiasedly measures the causal effect of front page display advertising

exposure on the tendency to search for brand j within the ten minute time window. This

estimate would be biased if ADit were correlated with εijt, but this would require users to

be systematically more or less likely to search when they visit the front page on an even

second relative to an odd second. On the other hand, we might underestimate the total

impact by not considering all searches caused by the display ad. For example, if a user’s first

exposure continues to influence their searching behavior all day long, and not just during the

limited ten-minute window, we will underestimate the impact of the ad on search behavior

by ignoring later searches caused by the ads and misattributing some of those searches to

the baseline search behavior.

We estimated standard errors by clustering the errors on the user level to account for

random effects and any autocorrelation between periods. For example, a user who sees a

headline of interest on the front page might spend an entire day researching that topic, and

therefore, may be less prone to search in the advertised category (or any category).

We are interested in understanding the impact of a full day of advertising on Yahoo!’s

Front Page, www.yahoo.com, not the impact of the split. We do this for expositional sim-

plicity. All of our main estimates below present the results for the counterfactual, “What

would the ad effect have been had the advertiser purchased the full day of advertising?”

We compute these estimates simply by multiplying the estimates by the total number of

impressions shown from both advertisers (i.e., twice the impressions shown in an ad split).

We pause to point out that wearing out of the effects, which could complicate this simplified

counterfactual computation, are found to be of modest importance in the next subsection;

12

for now, we focus on the conceptually simpler target versus control comparison: “What did

people search for after seeing one display ad rather than the other?”

Table 3 contains the results from estimating equation (1) for each of the three ad cam-

paigns. We have transformed the linear probability model’s probability estimates to report

the display ad’s impact in terms of number of searches for the advertised brand and for

any of its competitors’ brands. We also separately report estimated effects for the competi-

tors’ brands with the four highest t-statistics. The baseline estimates, representing the total

number of searches by all visitors that would have occurred if only the control display ad

were delivered, are the estimated intercepts multiplied by the total number of impressions

delivered on that day. The second column of estimates shows the estimated number of incre-

mental searches over the baseline from showing the target display, computed by multiplying

the estimated lifts in the propensity to search with the total number of impressions deliv-

ered. Both t-statistics based on the OLS standard errors and clustered standard errors are

reported by each of the estimates.

In all three campaigns, we found a statistically significant lift in searches for the advertised

brand from its display ad (p < 0.001 for all three campaigns). Both Acura and Galaxy Tab ad

campaigns caused a 44% lift in searches for their brands while the Progressive ad campaign

only caused a 28% lift in searches for its brand. If the advertised brands decided to run their

campaigns for the full day then Samsung would have had 424 more searches, Acura would

have had 1, 555 more searches, and Progressive would have had 1, 135 more searches relative

to not advertising.

Not only did we find search lifts for the advertised brand but also for its competitors.

Two of the three campaigns caused significant increase in searches for their competitors.

The Samsung Galaxy Tab ad campaign caused a 6.0% increase in searches for any of its

competitors. Most of this increase was due to searches for Apple’s iPad. If the Galaxy Tab

ad campaign were run for the full day then it would have produced 857 more searches for

the iPad. This number is about two times larger than that for the Galaxy Tab, 424. In

13

essence, at least for driving online searches, the Galaxy Tab ad campaign was more of an

advertisement for the iPad than it was for the Galaxy Tab. The other product that had a

statistically significant increase in searches from the Galaxy Tab display ad was the Motorola

Xoom. Though not as big of an increase as that of the iPad in absolute terms, the percentage

lift for Xoom searches was 22.8%.

Acura’s ad campaign also caused an increase in searches for its competitors, though the

estimated effect was smaller at 3.0%. Note that the relative sizes of the baselines matter–with

36 competitors to account for, some of which are much larger than Acura–the total search

impact for all the competitors combined, in terms of number of searches, is approximately 7.7

times larger than for the advertiser, Acura. Again, we see that advertising produced more

searches for its competitors than for its own brand. The brands with the largest increase in

absolute terms are Volkswagen, Hyundai, Lexus, and Volvo with effects ranging from 478 to

894 more searches.

In contrast, the Progressive Auto Insurance ad campaign did not noticeably increase

searches for its competitors. A possible reason for this is that auto insurers’ websites also

quote their competitors’ prices. Both the creative in the campaign and Progressive’s website

suggest that Progressive will present users with prices from its competitors. Therefore, users

do not need to incur the cost of searching for information about competitors if they obtain

that information from the advertised brand.

These differences in search behavior are for Yahoo! Search only. We would expect users

to perform any incremental searches using their preferred search engine: whether visiting a

search engine’s website, typing in the search box in their web browser, or using a browser

toolbar. Holding the search channel constant, we would expect to find a similar relative

composition of findings across search methods.

14

6.2 The Number of Exposures as a Moderator

Our second analysis considers the number of ad exposure as a moderator to the effects

that we measured in the previous section. Past research has shown that the frequency of

ads can affect the effectiveness of display ads. For example, Lewis (2012) shows a decrease

in clicking on the display ad after multiple exposures of the ad. As a result, we hypothesize

that the number of ad exposures delivered to a user should also moderate the display ad’s

effect on search.

This is important to publishers. In the CPC business model of display advertising,

rotating display ads might be profit maximizing in the presence of decreasing marginal

effects on clicking.17 However, publishers should also consider possible display advertising

effects on search when making this decision. More exposure will reduce the likelihood of

clicking on the display ad but may increase the likelihood of searching for competing brands

or the display advertised brand, which can turn into potential search revenue.

To test for moderation, we estimated a more general empirical model than that in our

previous section. For each advertising campaign and for two separate outcomes, searches

for the advertised brand and searches for any of the competitors’ brands, we estimated the

following linear probability model:

Searchijt = αj +(β0j + β1jExposit + β2jExpos

2it

)ADjt + γ1jExposit + γ2jExpos

2it

+ θ1jTotalV isitsi + θ2jTotalV isits2i + τ1jt+ τ2jt

2 + εijt

(2)

We denote subscripts for user i, brand j, and time period t. Searchijt is an indicator equal

to one if user i searched for brand j during period t. Exposit is number of times user i was

exposed to the target ad at time t. ADit is an indicator equal t one if user i is delivered the

target ad at time t. TotalV isitsi is user’s i TotalVisits to the front page during the day of

the ad campaign. Finally, εijt is the error term clustered at the user level.

17The display advertisements that we study do not use the CPC model, but advertisers pay a fix fee forthe display ad space. Although Yahoo! does not price display ad on the front page based on clicks, the CPCmodel is a common way to price display ads.

15

To properly quantify the impact of multiple ad exposures, we have to control for activity

bias and time trends. In general, the number of exposures and the type of people who do

a lot of activities on the internet will be correlated (Lewis, Rao, and Reiley 2011). We

added proxy for those differences by including the total number of visits to the front page

in our model. If one does not control for activity bias, the estimated marginal effect from

showing an additional impression to a user on the propensity to search will be confounded

by the difference in the likelihood to search between users of varying browsing and searching

behavior. We also added a time trend to our model to control for time of day effects. Because

the number of impressions increments over the course of the day, the number of exposures,

Exposit, will be correlated with the time trend. After controlling for these two potential

confounds, we are able to identify the effects of ad frequency, Exposit, through the natural

randomization from the ad split.18

In addition, we mentioned in previous sections that our estimate of the total impact of

a display ad on search underestimates the true impact because the effect could spill over

beyond the ten minute window. As a robustness check, we added Exposit not interacted by

ADit to capture spillovers into the control group.

Table 4-6 contain the results of estimating variations of equation (2). Numbers in paren-

thesis are t-statistics using clustered errors, and estimates in bold are significant at the 5%

level. Panel A shows the estimates predicting the propensity to search for the advertised

brand. Panel B provides the analogous estimates for the competitors’ brands. From left to

right in each panel, we start with regressing the dependent variable only on ADit, essentially

the same model as in equation 1, and end with our full model, presenting several interesting

intermediate estimates. As written in equation (2), we added a quadratic component to each

of our independent variables to capture any diminishing marginal effects.

Table 4 contains the results from the Samsung Galaxy Tab ad campaign. The estimates

18This model does not allow us to account for variation in the marginal return to frequency as in theuser-level model found in Lewis (2012). Here, we have adopted an impression-level model to leverage thericher variation in the time-stamped impression and search data. This allows us to harness more of theimpression-level variation in frequency and measure its temporal correlation with search.

16

show no evidence that the number of exposures moderates the effect on the propensity to

search for Samsung Galaxy Tab. All coefficients relating to the number of ad exposures are

insignificant. However, there appears to be an increasing marginal effect on the propensity to

search for Samsung’s competitors’ brands because the coefficient on the interaction between

the target ad exposure indicator and the number of exposures is significantly positive. This

positive coefficient means that a user is increasingly more likely to search for a competitor’s

brand in the number of target ad exposures. However, no similar effect is found in the

number of control ad exposures. This suggests that there is minimal spillover of a display

ad’s effect on search into subsequent control groups.

Table 5 presents the results from the Acura ad campaign. For the effects on searches for

Acura, we find that the number of exposures does moderate the effect on online searching

for Acura. Focusing on the significant estimates, the results suggest that the marginal effect

increases up to the seventh impression and diminishes afterwards. In contrast, there is no

evidence that the number of exposures moderates the effect on online searches for Acura’s

competitors.

Finally, Table 6 presents the results from the Progressive ad campaign. Like the Acura

campaign we find that the number of exposures moderates the impact on searching for

Progressive. Significant coefficients suggest that the marginal effect increases up to the

sixteenth exposure, and diminishes afterwards. There was no effect from display advertising

on searching for Progressive’s competitors, which is consistent with our primary results for

Progressive using the model in equation 1 found in Table 3.

Overall, the campaigns showed heterogeneous effects of the number of exposures on

search. The analysis of the Samsung Ad campaign suggests that there is wear-in for the

display ad impact on searches for the competitors’ brands, but the analyses of the Acura cam-

paign and Progressive campaign ruled out significant moderation of the effect on competitor-

branded searches. Furthermore, we found that the number of exposures both increases and

decreases the effects on searches for the display advertised brand within an advertising cam-

17

paign and across advertising campaigns. Given the evidence of little moderation by the

number of exposures, publishers might need to consider display ad effects on search when

deciding to rotate display ads to users even though they observe stronger decreasing of

display ad clicking.

6.3 Robustness Check

As a robustness check, we looked at the most statistically impacted words, queries, and

domains clicked. The results of this robustness check parallel those from our brand analysis,

but they also provide more insights that we have not captured. As expected, the display

ad caused an increase in clicking on the brands’ domains from both organic and sponsored

links. For example, the Acura display ad increased clicking on links to Acura.com, VW.com,

Hyundai.com, Lexus.com, and Volvocars.com, and the Galaxy Tab’s display ad increased

clicking on links to store.apple.com. The t-stats for these increases were greater than 3.

Interestingly, the display ad also statistically significantly increased clicks on search links for

trade, review, and distribution websites. For example, Acura’s display ad increased clicks for

Motortrend.com, Edmunds.com, and autobytel.com, and Galaxy Tab’s display ad increased

clicks for reviews.cnet.com, besttablet2011.com, and bestbuy.com. The increase in clicks

for these “market information” websites speaks to the literature that suggest advertising

invites and motivates consumers to search for more information. This is true not only for

information about the advertised brand but also about the whole product market.

We also looked at the most positively impacted queries and words. For the Progressive

Ad, the most impacted query was “progressive flo.” Flo is the name of the woman in the

Progressive Ad. This implies that a large portion of the estimated search lift for Progressive

was searches intended for Flo and not for Progressive Auto Insurance. When looking at the

most impacted search words and queries due to the Acura ad, we find that the Acura ad

increased searches for specific car models besides just the make of the cars. For example, the

Acura ad statistically significantly increased searches for Hyundai Sonata, Ford Flex, Nissan

18

Optima, and Honda Crosstour. Overall, these checks echo the same story that is suggested

by our previous results.

7 Economic and Strategic Effects of Extra Searches

7.1 The Publisher’s Extra Search Revenue

Using the estimates in Table 3, we convert these extra searches to revenue for the pub-

lisher. In search advertising, an advertiser pays the publisher every time a user clicks on the

ad. Ideally, to calculate the extra search revenue generated by display advertising, we would

need to know which searches resulted in an ad click and the amount paid by the search

advertiser. We do not have this data, but instead, we used average CTR and CPC data

from Microsoft Ad Center.

Ad Center is where advertisers and media planners buy ad space on Yahoo!’s search

result pages. Ad Center includes tools that help advertisers estimate the daily cost of their

search advertising campaigns. For each keyword, Microsoft Ad Center gives the advertiser

a recommended list of related keywords, average daily searches, CTRs, and average CPCs

paid by advertisers. Click through rates measure the probability that a user clicks on an

ad, and average CPC is what the advertiser expects to pay the publisher for that click. We

multiply the click through rate and the average CPC to approximate the expected revenue

from one search.

For each brand, we have a recommended list of keywords from Microsoft Ad Center

and their corresponding click through rates and average CPCs. We excluded from this list

keywords without the brand’s name, and with the remaining keywords, we calculated a

weighted average of the click through rates and average CPCs to obtain the brand level click

through rate (CTR) and CPC. Each keyword’s average CPC and click through rate were

weighted by the average daily searches for that keyword.

Tables 7-9 are the estimated search revenues generated by one day of display advertising

19

for each of the advertising campaigns. Columns 1-3 are the estimated search lifts and their

corresponding 95% confidence interval from Table 3; columns 4-6 are the numbers use to

calculate the expected revenue for a branded search; column 7-9 are the publisher’s revenue

generated by the display ad from our considered keywords along with 95% confidence interval

bounds. We calculated the publisher’s revenue by multiplying the estimated search lift with

expected revenue for the branded search.

Based on the point estimates, all three ad campaigns generated search revenue for the

publisher. A full day of the Progressive display ad would have generated 107.32 USD of search

revenue; for Acura and Samsung, the corresponding figure is 1, 037.07 USD and 96.30 USD,

respectively. From these overall point estimates, all of which are statistically significant at the

5% level, we see that some of the extra revenue was offset by a loss of revenue because display

advertising caused some users to search for one brand instead of another brand. However,

from Table 3, we found that none of the negative search lifts were statistically different from

zero and that the collective estimate of searches for all competitors’ brands was positive and

statistically significant. These facts lead us to believe that display advertising has an overall

positive effect on search revenue.

Because visitors of the Front Page use other search engines, we expect that the display

ads not only increase Yahoo!’s search revenue but also that of other search engines. As of

July 2011, Yahoo only claimed a 16% share of the U.S. search market according to comScore,

meaning that this analysis is potentially ignoring spillovers to other search engine publish-

ers.19 Using the 16% market share as one bound, we would suspect that the number of

spillovers to other search engines may be as large as five times these estimated figures. Yet,

visitors to the Yahoo! Front Page may not be perfectly average customers. As a lower bound

on the spillovers to other search engines, we consider the fact that loyal Yahoo! Toolbar users

perform 50% of their searches on Yahoo!.20 This implies that the number of spillovers to

19“comScore Releases July 2011 U.S. Search Engine Rankings,”http://www.comscore.com/Press Events/Press Releases/2011/8/comScore Releases July 2011 U.S. SearchEngine Rankings, retrieved Feb. 21, 2012.

20This number was derived from the same search study found in Lewis, Rao, and Reiley (2011) where an

20

other search engines may be as small as our estimated figures. Therefore, the collective rel-

ative spillover to other search engines is somewhere between one and five times the revenue

gained by Yahoo! Search from the Front Page display ads.

Furthermore, our focus on branded searches within a category led us to omit many

relevant and valuable search terms. As such, we have likely omitted significant revenue

from those searches. However, the practical, technical, and statistical challenges involved

in identifying and measuring all relevant search terms’ incremental search impact from the

display ads is well beyond the scope of this paper and is the topic of future research. Limiting

our attention to the brand-specific keywords will lead us to understate the total impact of

the campaign, but it gives us a more statistically precise comparison among competitors.

7.2 The Advertiser’s and its Competitors’ Extra Revenue

We have shown that display advertising increases searches for many brands in the ad-

vertised product category, but to calculate the profit or revenue impact, we would need to

know the advertiser’s marginal revenue from a search ad impression. The marginal revenue

should be at least the CPC. However, we are unable to link transaction data for these ad-

vertisers with the search data and, hence, cannot systematically calculate the effects on the

advertiser’s revenues nor its competitors.21 We posit that the relative effects on sales for

the advertisers and competitors may mirror the relative effects for searches; however, future

research should investigate this hypothesis.

additional unpublished analysis of that Front Page campaign showed that during that day, 133,717 Yahoo!Toolbar users performed a search on Yahoo out of a total number of 191,134 unique users who performed atleast one search on Yahoo!, Google, or Bing. This produces a ratio of 70%; however, many users actually usedmultiple search engines: in total, there were 234,509 unique users X search engine observations, suggestingthat a ratio of 57%. Given the skew of the Y! Toolbar, the uncertainty regarding generalizing this numberfrom a small fraction of exposed users, and for ease of exposition, we approximate the fraction of searchesaccounted for by Yahoo! Search for visitors to Yahoo!’s Front Page as roughly 50%.

21Other research (Johnson, Lewis, and Reiley 2012; Lewis and Rao 2012) that has been able to link adexposure to ad exposure has found quantifying marginal returns to advertising for larger retailers a verychallenging problem due to the inherently low signal-to-noise nature of cost-effective advertising.

21

7.3 The Complementarities between Display and Search Adver-

tising

Even though we do not measure the revenue generated by these spillovers, the spillovers

do create complementarities between display and search advertising through the cost of

advertising. This is due to the generalized second price auction. The Appendix presents

a simple model and derives necessary conditions for display and search advertising to be

complements. Our findings mainly depend on two necessary conditions. Our empirical

finding satisfies one condition, and another condition is that a search ad’s CPC is weakly

increasing in the desired click through rate. The second condition is a property resulting

from the generalized second-price auction. In equilibrium of the auction, ads at the top of

the page, a higher click through rate spot, cost more than ads at the bottom of the page.

The intuition regarding the complementarities of display and search ads is simple: if

display advertising increases the number of searches, then holding revenues constant by

holding the number of clicks constant permits an advertiser to bid for a lower CTR. Lowering

the CPC directly translates into a larger profit margin for each search click; hence, display

and search ads are complements. A very similar calculation and logic applies to a firm’s

display ads and the competitors’ search ads because the same opportunity to reduce the

CPC applies as much to the competitors as to the display-advertising firm. This larger

profit margin for competitors’ search ads implies that display and search ads are strategic

complements.

To illustrate our point, we use the bids on 12 February 2012 for the branded keywords

of the three advertisers and calculated the CPC for each position. Figure 5 shows the

relationship between CPC and CTR on the search result pages for the three advertised

brands: Progressive, Acura, and Samsung. Due to confidentiality, CPC are in terms of the

percentage of the top position, position 1. CTRs for the four search ad positions are averages

of CTRs from a sample of queries with at least four ads that are found in Reiley, Li, and

Lewis (2010). They are also expressed in terms of percentage of the top position. For an

22

example of the cost savings, let’s consider searches for Acura. If an advertiser decided to

move from position 2 to position 3, the advertiser lowers its CPC by 27%, but loses 30%

clicks by moving to a lower CTR position. However, the 44% increase in searches due to

Acura’s display ad on the front page will make up the loss in clicks from the lower CTR.

Therefore, the advertiser in position 2 can save 27% per click for the same number of clicks

by moving down to position 3 when Acura has a display ad on Yahoo!’s front page.

8 Conclusion

The three empirical studies found that display advertising increases searches for the

advertised brand. We also showed the increase in searches created complementarities between

display and search advertising. That said, there might be other complementarities that we

cannot measure. For example, if the advertiser chooses to advertise on its search result

page, the display advertiser can reinforce the marketing message, provide more product

information, and advertise distribution channels. As a result of these synergies between

search and display advertising, Naik and Raman (2003)’s theoretical model would suggest

that advertisers should invest more in display advertising than if there were no complements.

Similarly, we also found that display advertising can increase the searches for the com-

petitors’ brands. Because competitors get the same benefits as the display advertiser for

the increase in search, display advertising is also a strategic complement to a competitor’s

search advertising. When such strategic complements exist, a brand can benefit from know-

ing when its competitors are going to advertise by adjusting their search advertising bids

during those times. Again, there might be other strategic complementarities that we cannot

measure. For example, Burke and Srull (1988) show that consumers’ recall of an ad’s brand

information is hindered by exposure to a competitor’s ad. So, if a display ad causes users to

search, a prominently displayed search ad for a competitor can attenuate the recall of the

display advertiser’s marketing message. This is another incentive for an advertiser to run a

23

search ad during its competitors display ad campaign.

We converted these effects to extra search revenue for the publisher who owns the search

engine. We found positive effects on publisher’s search revenue from display advertising.

This means a publisher who owns a display advertising channel should also own a search

advertising channel to capture all the revenue generated by display advertising. Like all

complements in oligopolistic markets, the market structure where publishers own both chan-

nels is the social welfare-maximizing outcome (see Economides and Salop (1992) for further

discussion). This is because the publisher will internalize these complements, which will

cause them to lower prices for display advertising. In other words, it is best for advertisers if

publishers compete on multichannel online advertising systems instead on individual online

advertising channels. In fact, this is for the most part how the current online advertising

market in the US is organized. The top US online advertising firms, Google, Yahoo!, and

Microsoft, all own both search and display advertising channels.

In our final analysis, we showed how these complements are moderated by the number

of ad exposures. We found that the number of exposures mildly moderated the effects

of display advertising on search. These results are important to advertisers because they

need to understand the marginal effectiveness of their ads both with respect to consumer

awareness of their brand and the awareness spillovers to their competitors. These results

are also important to a search and display publisher who uses the CPC model for display

advertising because it will affect the way that they rotate display advertisement to users.

Our analysis has limitations. First, we are unable to analyze how these search effects

will translate to revenues for the advertisers and their competitors. One way to do this

would involve randomizing firms to conduct display advertising, search advertising, and

both and linking that ad exposure data to purchase data.22 Second, we observed significant

heterogeneity in the effects across ad campaigns and within an ad campaign across brands.

22In practice, such an experiment has meaningful challenges involving measuring a second derivativeof the impact of display advertising on the marginal profitability of search advertising. Obtaining such ameasurement is a herculean task amid the signal-to-noise reality of advertising effectiveness.

24

A next step would be to understand what drives these differences. For example, a brand’s

current goodwill and past investment in advertising can be possible moderators and explain

the heterogeneity. Third, we also were unable to analyze equilibrium prices for display ads.

It would be interesting to systematically compare display ad prices between publishers who

only own display ad channels and those who own both display ad and search ad channels.

Based on our empirical findings and the theory about complement products, we expect the

price of the former to be higher than the latter, holding all else equal. Finally, our results

can be a proxy for the perceptual distances between brands. Future research could explore

the use of search response to ads to form a perceptual map of brands and calculate the

substitutability between brands when purchase data is unavailable.

In conclusion, as a larger share of the media planner’s budget is invested in online ad-

vertising, researchers are seeking to understand how to achieve the optimal mix of online

advertising channels that maximizes the return for every online advertising dollar. Our paper

touches on this issue by showing how two of the largest online advertising channels interact

with each other: display and search. Specifically, our results showed that a brand’s ad might

have unintended effects across the advertising channels. We hope these results will guide

researchers and practitioners in their goal of optimally allocating advertising dollars and

creating the most effective marketing message possible for the advertiser.

References

“Attitudes to Internet and New Media Marketing.” Mintel Reports database 2010, September

8, 2011, http://academic.mintel.proxy.uchicago.edu.

“Autobytel - New Cars.” Autobytel, June 30, 2011, http://www.autobytel.com/new-cars.

“Auto Insurance.” Mintel Reports database 2009, August 1, 2011,

http://academic.mintel.proxy.uchicago.edu.

Bulow, Jeremy I., John D. Geanakoplos, and Paul D. Klemperer 1985. “Mulimarket Oligopoly:

25

Strategic Substitutes and Complements.” Journal of Political Economics, 93:488-511.

Burke, Raymond R. and Thomas K. Srull 1988. “Competitive Interference and Consumer

Memory for Advertising.” Journal of Consumer Research, 15:55-68.

Chang, Yuhmiin and Esther Thorson 2004. “Television and Web Advertising Synergies.”

Journal of Advertising, 33:75-84.

Economides, Nicholas and Steven C. Salop 1992. “Competition and Integration Among

Complements and Network Market Structure.” The Journal of Industrial Economics

40:105-123.

Edelman, Benjamin, Michael Ostrovsky, and Michael Schwarz 2005. “Internet Advertising

and the Generalized Second Price Auction: Selling Billions of Dollars Worth of Key-

words.” NBER Working Paper No. 11765.

Evans, David S. 2009. “The Online Advertising Industry: Economics, Evolution, and Pri-

vacy.” Journal of Economic Perspectives, 32:37-60.

Franklin, Eric 2011. “CNET looks at current and upcoming tablets.” CNET, July, 29, 2011,

http://news.cnet.com/8301-17938 105-20037960-1/cnet-looks-at-current-and-upcoming-tablets/.

Goldfarb, Avi and Catherine Tucker 2010. “Search Engine Advertising: Channel Substitu-

tion when Pricing Ads to Context.” NET Institute Working Paper No. 07-23.

Johnson, Garrett, Randall Lewis, and David Reiley 2012. “Location, Location, Location:

Proximity and Repetition Increase Effectiveness of Display Ads in Controlled

Experiments,” working paper.

Joo, Mingyu, Kenneth C. Wilbur, and Yi Zhu 2011. “Does Television Advertising Influence

Online Search?” Working Paper, Duke University.

Lewis, Randall 2012. “Worn-Out or Just Getting Started? The Impact of Frequency in

Online Display Advertising,” working paper.

Lewis, Randall and Justin Rao 2012. “On the Near Impossibility of Measuring Advertising

Effectiveness,” working paper.

Lewis, Randall, Justin Rao, and David Reiley 2011. “Here, There, and Everywhere: Cor-

26

related Online Behaviors Can Lead to Overestimates of the Effects of Advertising,”

20th International World Wide Web Conference (WWW20), April 2011.

Lewis, Randall and David Reiley 2012. “Does Retail Advertising Work? Measuring the

Effects of Advertising on Sales via a Controlled Experiment on Yahoo!,” working

paper.

Mayzlin, Dina and Jiwoong Shin 2011. “Uninformative Advertising as an Invitation to

Search.” Marketing Science, 30:666-685.

Menon, Satya and Dilip Soman 2002. “Managing the Power of Curiosity for Effective Web

Advertising Strategies.” Journal of Advertising, 31:1-14.

Naik, Prasad A. and Kalyan Raman 2003. “Understanding the Impact of Synergy in Mul-

timedia Communications.” Journal of Marketing Research, 40:375-388.

Nedungadi, Prakash 1990. “Recall and Consumer Consideration Sets: Influencing Choice

without Altering Brand Evaluations.” Journal of Consumer Research, 17:263-276.

Reiley, David, Sai-Ming Li, and Randall Lewis 2010. “Northern exposure: a field experi-

ment measuring externalities between search advertisements,” Proceedings of the 11th

ACM Conference on Electronic Commerce (EC’10), pp. 297-304.

Swasy, John L. and Arno J. Rethans 1986. “Knowledge Effects on Curiosity and New Prod-

uct Advertising.” Journal of Advertising, 15:28-34.

Appendix: Search & Display as Complements and Strate-

gic Complements

Consider the following model for a firm’s profits as a function of the quantity of adver-

tising:

Π (Ad, As) = Ad · vd + As · vs − Ad · Pd − As · Ps (As/Qs (Ad))

27

where Ad and As are the quantity of display and search advertising denoted in impressions

and clicks, respectively, and Qs is the total number of searches. v and P refer to the

expected value and price of the advertising, A. In the case of display advertising, Pd would

be denoted in cost per mille (CPM), meaning the price of 1,000 display impressions. For

search advertising, the generalized second-price auction is used to sell search ad placements

on search result pages. In the auction, an advertiser determines a desired click-through rate

(CTR) and bids sufficiently high to achieve the ranking that yields that CTR. Here, we

write Ps is the average price paid to obtain a CTR of As/Qs (Ad). A necessary condition for

the existence of an equilibrium in the auction requires that the price paid is increasing in

the CTR, P ′s > 0. In order for search and display advertising to be complements, marginal

profits from search advertising must be increasing in the volume of display advertising. We

derive the cross partial derivative of the profit function below.

∂Π

∂As

= vs − Ps

(As

Qs (Ad)

)− As · P ′

s

(As

Qs (Ad)

)· 1

Qs (Ad)

∂2Π (Ad, As)

∂As∂Ad

=Q′

s (Ad)

Qs (Ad)2

((1 + As) · P ′

s

(As

Qs (Ad)

)+

As

Qs (Ad)· P ′′

s

(As

Qs (Ad)

))> 0

A key finding from our empirical work reveals that Q′s > 0: we find that the number of

searches, Qs (Ad), is an increasing function of display advertising, Ad. The other necessary

conditions for search and display advertising to be complements are that P ′s > 0, which

follows from the auction’s structure, and P ′′s > −1+As

As·Qs (Ad) · P ′

s

(As

Qs(Ad)

), which bounds

the degree of concavity for the average price paid as a function of the CTR.

The same derivation used for complements now applies to strategic complements. To

simplify notation, we write the profit function with only the relevant strategic variables:

Πj(Ai

d, Ajs

)= Aj

s · vjs − Ajs · P j

s

(Aj

s/Qjs

(Ai

d

)).

We write i for the display advertiser and j for competitors. As the strategic variables, we

28

consider the quantity of display advertising, Aid, and the competitor’s search advertising,

Ajs. Again, P j

s is the average price paid per click to achieve the CTR of Ajs/Q

js (Ai

d) and is

increasing for all market participants: P ′s > 0. From the empirical results we know that the

number of competitors’ searches is increasing in the firm’s display advertising, Qj′s (Ai

d) > 0.

Given these identical comparative statics, the strategic complementarity of display and search

advertising follows from applying a superscript j to all variables other than Ad, to which one

applies an i superscript:

∂2Πj (Aid, A

js)

∂Ajs∂Ai

d

> 0.

29

Appendix: Tables and Figures

Figure 1: Snapshot of the Frontpage

30

Table 1: Creatives for Ad Campaigns

Date of Ad Split Target Ad Control Ad

11 January 2011

10 February 2011

29 June 2011

31

Table 2: Summary Statistics

!"#$"%&'()"*+),-"-$(-$.(/#01#'(($2')

34-0)5*(4#"*.'3.4#")6,7

,"8(4*19():"&";<)6"%

="-')0>)3+),?&$- !"##$"#$## !"##$"!$#" !"##$"%$!&

,"8?&'),$@'(

'()*+,-./012,(3,4567.1,8696)(29 :";%<=;%>< :#;=#=;>=% =<;%!";=#>

'()*+,-./012,(3,8696)9 #<#;&?=;==# #<#;?&=;<># #%#;:%";!""

'()*+,-./012,(3,@AB(9.219,)(,)C1,'*2D1),EF >%;#?!;<<& >?;%>:;&#: >";>%%;&"=

/'#.'*-"1')0>)A('#()BC0),'"#.C'+)>0#)D'&'2"*-)E'<F0#+( "G"%H "G>"H "G":H

60-"&)G48%'#)0>)!$($-()?'#)A('#

I1*5 :G!= :G#? :G!&

J)*5F*2F,K1L6*)6(5 %G?& %G?= %G%:

I656/./ #G"" #G"" #G""

!?)C,M12N15)6+1 #G"" #G"" #G""

?")C,M12N15)6+1 !G"" !G"" !G""

<?)C,M12N15)6+1 ?G"" ?G"" ?G""

I*A6/./ :;&"!G"" :;>&%G"" :;>&<G""

60-"&)G48%'#)0>)H;?0(4#')-0)-C')6"#1'-)3+)?'#)A('#

I1*5 !G#! !G"< !G#?

J)*5F*2F,K1L6*)6(5 =G:? =G:= =G?"

I656/./ "G"" "G"" "G""

!?)C,M12N15)6+1 #G"" #G"" #G""

?")C,M12N15)6+1 #G"" #G"" #G""

<?)C,M12N15)6+1 =G"" =G"" =G""

I*A6/./ !;::>G"" !;:?=G"" !;::<G""

32

Figure 2: Distribution of the Total Number of Visits

Number of Impressions Delivered

Den

sity

0.00

0.10

0.20

0.30

1 3 5 7 9 12 15 18 21 24 27 30

Ad Split on 2011/01/11


Den

sity

0.00

0.10

0.20

0.30

1 3 5 7 9 12 15 18 21 24 27 30



Den

sity

0.00

0.10

0.20

0.30

1 3 5 7 9 12 15 18 21 24 27 30


33

Figure 3: Distribution of the Total Number of Exposures to the Target Ad


Den

sity

0.00

0.10

0.20

0.30

0 2 4 6 8 11 14 17 20 23 26 29

Progressive Auto Insurance


Den

sity

0.00

0.10

0.20

0.30

0 2 4 6 8 11 14 17 20 23 26 29

Acura TSX


Den

sity

0.00

0.10

0.20

0.30

0 2 4 6 8 11 14 17 20 23 26 29

Samsung's Galaxy Tab

34

Figure 4: Distribution of the Total Number of Exposures to the Target Ad Given Users WhoOnly Visited the Front Page 10 times


Den

sity

0.00

0.05

0.10

0.15

0.20

0.25

0 1 2 3 4 5 6 7 8 9 10 11

Progressive Auto Insurance


Den

sity

0.00

0.05

0.10

0.15

0.20

0.25

0 1 2 3 4 5 6 7 8 9 10 11

Acura TSX


Den

sity

0.00

0.05

0.10

0.15

0.20

0.25

1 2 3 4 5 6 7 8 9 10 11

Samsung's Galaxy Tab


Den

sity

0.00

0.05

0.10

0.15

0.20

0.25

0 1 2 3 4 5 6 7 8 9 10 11

binomial(10, .5)

35

Table 3: Results of the Effects of Advertising on Search

Control Search Lift from Advertising

Searches Estimate OLS T-stat Cluster T-stat Estimate OLS T-stat Cluster T-statPercentage

LiftCompetitor/

Own

Samsung Galaxy Tab Advertising Campaign

Samsung Galaxy Tab 958 19.78 20.57 424 6.20 6.32 44.3% 1.00

All Competitors 16,662 89.87 82.42 994 3.79 3.81 6.0% 2.34

Apple Ipad 9,851 68.64 63.21 857 4.23 4.25 8.7% 2.02

Motorola Xoom 663 17.23 16.74 151 2.79 2.79 22.8% 0.36

Blackberry Playbook 317 11.92 11.34 71 1.89 1.90 22.4% 0.17

Viewsonic 18 2.55 3.00 14 1.39 1.39 77.2% 0.03

Acura Advertising Campaign

Acura 3,539 38.12 38.34 1,555 11.84 11.78 43.9% 1.00

All Competitors 401,927 445.80 389.84 12,035 9.43 9.44 3.0% 7.74

Volkswagen 5,840 52.12 48.24 894 5.64 5.62 15.3% 0.58

Hyundai 5,399 50.05 46.94 853 5.59 5.55 15.8% 0.55

Lexus 3,907 42.54 39.37 631 4.86 4.85 16.2% 0.41

Volvo 2,183 31.39 29.31 478 4.86 4.75 21.9% 0.31

Progressive Auto Insurance Advertising Campaign

Progressive 4,104 42.41 42.76 1,135 8.30 8.34 27.6% 1.00

All Competitors 23,035 106.84 99.34 327 1.07 1.08 1.4% 0.29

Allstate 2,968 38.09 36.52 124 1.12 1.13 4.2% 0.11

USAA 7,870 62.30 56.97 187 1.05 1.06 2.4% 0.17

Safeco 214 10.01 9.73 29 0.96 0.97 13.6% 0.03

Nationwide Insurance 880 20.64 19.81 54 0.90 0.91 6.2% 0.05

36

Table 4: Results of the Effects of the Number of Exposures on Search from the SamsungGalaxy Tab Ad Campaign

Advertised Brand Competitors' Brands

[1] [2] [3] [4] [5] [6] [7] [8]Samsung Galaxy Tab(N = 160,415,996)

AD 2.66E-06 2.29E-06 2.67E-06 2.41E-06 6.06E-06 3.22E-06 7.57E-06 2.73E-06(6.36) (5.43) (5.00) (2.87) (3.74) (1.98) (3.88) (1.07)

AD X Expos 1.04E-07 1.80E-06(0.34) (2.30)

AD X Exps2 -2.69E-09 -4.00E-08(-0.18) (-1.30)

Expos 3.63E-07 -1.09E-07 -1.39E-07 2.85E-06 -1.26E-06 -1.72E-06(4.81) (-0.27) (-0.39) (11.68) (-0.89) (-1.19)

Expos2 1.45E-08 1.47E-08 -4.04E-08 -4.31E-08(0.67) (0.82) (-0.73) (-0.74)

TotalVisits 6.37E-08 6.37E-08 1.23E-06 1.23E-06(1.14) (1.14) (4.61) (4.61)

TotalVisits2 -1.94E-09 -1.94E-09 -2.33E-08 -2.32E-08(-2.59) (-2.59) (-5.26) (-5.24)

t 4.72E-07 4.61E-07 4.71E-06 4.51E-06(2.43) (2.39) (5.82) (5.53)

t2 -7.99E-09 -7.71E-09 -4.87E-08 -4.29E-08(-1.85) (-1.78) (-2.83) (-2.45)

Intercept 5.96E-06 5.00E-06 3.59E-06 3.70E-06 1.04E-04 9.62E-05 7.61E-05 7.81E-05(20.55) (15.63) (7.98) (7.57) (82.54) (72.33) (40.05) (38.86)

Table 5: Results of the Effects of the Number of Exposures on Search from the Acura TSXSports Wagon Ad Campaign


[1] [2] [3] [4] [5] [6] [7] [8]Acura(N = 170,472,981)

AD 9.10E-06 8.21E-06 8.75E-06 6.26E-06 7.08E-05 9.19E-06 6.39E-05 7.64E-05(11.77) (10.58) (9.41) (5.31) (9.47) (1.22) (7.14) (6.37)

AD X Expos 1.02E-06 -4.38E-06(2.68) (-1.13)

AD X Exps2 -2.80E-08 6.11E-08(-2.15) (0.39)

Expos 8.88E-07 6.35E-07 3.22E-07 6.18E-05 7.69E-06 8.55E-06(7.66) (1.10) (0.54) (44.46) (1.15) (1.26)

Expos2 -4.62E-08 -4.23E-08 -1.13E-07 -7.15E-08(-2.42) (-2.16) (-0.39) (-0.24)



t 4.96E-07 4.07E-07 8.48E-05 8.54E-05(1.48) (1.21) (23.43) (23.40)

t2 -4.29E-09 -1.77E-09 -1.26E-06 -1.27E-06(-0.76) (-0.31) (-16.64) (-16.51)


37

Table 6: Results of the Effects of the Number of Exposures on Search from the ProgressiveAuto Insurance Ad Campaign


[1] [2] [3] [4] [5] [6] [7] [8]Progressive Auto Insurance(N = 170,834,016)

AD 6.61E-06 6.34E-06 6.42E-06 3.47E-06 1.77E-06 2.09E-06 2.21E-06 1.21E-06(8.30) (7.95) (7.04) (2.84) (1.00) (1.17) (1.08) (0.44)

AD X Expos 1.27E-06 2.64E-07(3.46) (0.33)

AD X Exps2 -4.10E-08 3.69E-09(-3.11) (0.12)

Expos 2.67E-07 2.31E-07 -2.07E-07 -3.14E-07 -6.55E-07 -6.44E-07(2.78) (0.41) (-0.35) (-1.28) (-0.49) (-0.47)

Expos2 -7.62E-09 3.28E-09 3.79E-08 2.81E-08(-0.44) (0.16) (0.75) (0.55)



t 3.12E-07 2.25E-07 -2.15E-06 -2.22E-06(0.97) (0.69) (-2.91) (-2.99)

t2 -7.32E-09 -4.89E-09 1.08E-08 1.28E-08(-1.46) (-0.96) (0.82) (0.98)


Table 7: Publisher’s Search Revenue Generated by the Samsung Galaxy Tab Display Ad

Search Lift Search Advertising Prices Publisher's Revenue[1] [2] [3] [4] [5] [6] [7] [8] [9]

Searches Estimate95% CI

Lower Bound95% CI

Upper BoundAverage

CTRAverage

CPC

Average Revenue

Per Search EstimateLower Bound

Upper Bound

Samsung Galaxy Tab 424 292 556 4.53% $1.71 $0.08 $32.79 $22.61 $42.97

Competitors

Apple Ipad 857 462 1,252 3.10% $1.97 $0.06 $52.32 $28.20 $76.44

Motorola Xoom 151 45 258 7.27% $0.73 $0.05 $8.06 $2.41 $13.71

Blackberry Playbook 71 -2 144 6.83% $0.91 $0.06 $4.40 -$0.14 $8.94

Vizio 14 -6 34 9.09% $0.19 $0.02 $0.24 -$0.10 $0.58

Toshiba 28 -16 71 5.52% $0.83 $0.05 $1.26 -$0.74 $3.26

Acer Iconia 41 -25 107 5.33% $0.97 $0.05 $2.13 -$1.28 $5.54

Archos 10 -21 40 7.85% $0.50 $0.04 $0.39 -$0.83 $1.60

Amazon Tablet 0 -24 23 2.16% $0.94 $0.02 $0.00 -$0.48 $0.48

Asus -18 -212 175 5.68% $0.60 $0.03 -$0.63 -$7.27 $6.02

HTC -33 -231 165 10.52% $1.01 $0.11 -$3.48 -$24.53 $17.58

Dell Streak -10 -59 38 8.98% $0.96 $0.09 -$0.90 -$5.11 $3.31

Sony -2 -11 7 5.29% $0.68 $0.04 -$0.07 -$0.39 $0.24

Micro Cruz -4 -12 4 5.22% $1.00 $0.05 -$0.21 -$0.62 $0.20

Coby -28 -67 10 5.99% $0.97 $0.06 -$1.65 -$3.90 $0.59

LG G-Slate -18 -39 3 4.15% $1.59 $0.07 -$1.19 -$2.58 $0.20

Viewsonic -40 -74 -6 6.62% $0.51 $0.03 -$1.36 -$2.52 -$0.20

Total 1,528 $96.30

38

Table 8: Publisher’s Search Revenue Generated by the Acura TSX Sports Wagon DisplayAd

Search Lift Search Advertising Prices Publisher's Revenue[1] [2] [3] [4] [5] [6] [7] [8] [9]


Lower Bound95% CI

Upper BoundAverage

CTRAverage

CPC

Average Revenue

per Search EstimateLower Bound

Upper Bound

Acura 1,555 1,297 1,814 5.88% $0.96 $0.06 $87.43 $72.89 $101.97

Competitors

Volkswagen 894 583 1,206 4.85% $1.26 $0.06 $54.76 $35.67 $73.85

Hyundai 853 552 1,155 3.91% $0.97 $0.04 $32.28 $20.87 $43.69

Lexus 631 376 886 4.40% $3.47 $0.15 $96.28 $57.35 $135.22

Volvo 478 281 676 4.24% $0.89 $0.04 $18.08 $10.61 $25.55

Subaru 521 298 743 5.12% $1.67 $0.09 $44.40 $25.40 $63.40

Honda 1,293 738 1,848 4.79% $1.21 $0.06 $75.13 $42.88 $107.38

Chrysler 699 383 1,014 4.64% $2.10 $0.10 $68.06 $37.33 $98.78

Mazda 488 244 732 4.32% $1.10 $0.05 $23.20 $11.62 $34.78

Nissan 809 401 1,217 5.65% $1.46 $0.08 $66.56 $32.99 $100.13

BMW 659 318 1,001 4.56% $2.27 $0.10 $68.29 $32.95 $103.63

Mercedes 439 166 712 5.69% $1.80 $0.10 $44.85 $16.92 $72.78

Dodge 748 239 1,257 3.59% $2.30 $0.08 $61.85 $19.79 $103.91

GMC 409 128 691 4.04% $1.68 $0.07 $27.82 $8.68 $46.97

Chevrolet 893 264 1,522 7.88% $1.18 $0.09 $83.03 $24.56 $141.50

Saab 156 43 269 3.61% $0.88 $0.03 $4.97 $1.38 $8.56

Mitsubishi 283 76 491 3.36% $1.69 $0.06 $16.06 $4.29 $27.83

Kia 558 149 968 6.68% $1.25 $0.08 $46.72 $12.45 $81.00

Cadillac 307 51 563 3.10% $1.92 $0.06 $18.27 $3.05 $33.49

Suzuki 219 23 415 4.89% $0.87 $0.04 $9.31 $0.96 $17.66

Land Rover 120 8 232 5.20% $0.76 $0.04 $4.74 $0.32 $9.16

Tesla 81 -2 164 2.81% $0.27 $0.01 $0.60 -$0.01 $1.22

Smart 404 -82 890 2.84% $1.06 $0.03 $12.11 -$2.45 $26.68

Infiniti 130 -28 288 6.43% $1.15 $0.07 $9.62 -$2.06 $21.30

Jaguar 119 -27 265 2.04% $1.27 $0.03 $3.10 -$0.69 $6.89

Buick 148 -68 364 2.84% $1.21 $0.03 $5.09 -$2.34 $12.53

Rolls Royce 37 -21 94 5.55% $0.41 $0.02 $0.84 -$0.47 $2.14

Lotus 73 -54 199 3.24% $0.56 $0.02 $1.32 -$0.98 $3.62

Audi 270 -304 845 3.29% $2.24 $0.07 $19.97 -$22.46 $62.40

Scion 45 -79 169 8.53% $1.15 $0.10 $4.39 -$7.81 $16.60

Porsche 46 -110 203 1.94% $1.79 $0.03 $1.61 -$3.84 $7.06

Toyota 201 -490 893 5.41% $1.49 $0.08 $16.25 -$39.58 $72.08

Fiat 23 -62 108 2.54% $0.77 $0.02 $0.45 -$1.21 $2.11

Jeep 67 -300 435 6.32% $1.59 $0.10 $6.77 -$30.27 $43.80

Ford 156 -1,277 1,590 4.79% $1.83 $0.09 $13.66 -$111.69 $139.02

Mini -32 -832 769 3.89% $1.13 $0.04 -$1.40 -$36.69 $33.90

Lincoln -325 -738 87 5.21% $0.85 $0.04 -$14.42 -$32.71 $3.87

Total 14,458 $1,032.07

39

Table 9: Publisher’s Search Revenue Generated by the Progressive Auto Insurance DisplayAd

Search Lift Search Advertising Prices Publisher Revenue[1] [2] [3] [4] [5] [6] [7] [8] [9]


Lower Bound95% CI

Upper BoundAverage

CTRAverage

CPC

Average Revenue

per Search EstimateLower Bound

Upper Bound

Progressive 1,135 868 1,401 2.08% $4.00 $0.08 $94.69 $72.44 $116.94

Competitors

Allstate 124 -91 338 2.71% $4.63 $0.13 $15.52 -$11.39 $42.42

USAA 187 -160 535 2.10% $0.33 $0.01 $1.30 -$1.11 $3.70

Safeco 29 -30 88 11.10% $0.76 $0.08 $2.45 -$2.50 $7.40

Nationwide Insurance 54 -62 171 10.85% $0.88 $0.10 $5.17 -$5.93 $16.27

AIG 36 -43 115 2.67% $0.49 $0.01 $0.47 -$0.56 $1.51

Geico 94 -136 324 2.59% $3.63 $0.09 $8.82 -$12.75 $30.39

Liberty Mutual 6 -91 102 11.97% $1.36 $0.16 $0.89 -$14.76 $16.55

Erie Insurance 1 -59 61 5.38% $1.37 $0.07 $0.08 -$4.32 $4.47

Travelers Insurance 0 -86 86 11.62% $1.26 $0.15 $0.00 -$12.56 $12.56

American Family Insurance -3 -66 60 10.02% $0.51 $0.05 -$0.16 -$3.40 $3.08

Farmer's Insurance -23 -152 107 2.36% $4.37 $0.10 -$2.33 -$15.69 $11.04

State Farm -125 -366 115 6.47% $0.72 $0.05 -$5.83 -$17.01 $5.34

21st Century Insurance -116 -234 3 12.62% $0.94 $0.12 -$13.75 -$27.81 $0.31

Total 1,399 $107.32

Figure 5: The Relationship between CPC and CTR on a Search Result Page!"#

$%&'(%)#*#Ad

Position 2 Ad

Position 3 Ad

Position 4

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

+,+-# *+,+-# .+,+-# /+,+-# 0+,+-# 1+,+-# 2+,+-# 3+,+-# 4+,+-# 5+,+-# *++,+-#

Perc

ent o

f Pa

ymen

t for

Top

Pos

ition

Percent of Click-Through Rate (CTR) of the Top Position*

Acura

Progressive

Samsung

* CTRs for the four search ad positions are averages for a sample of queries with at least four ads from Reiley, Li, and Lewis (2010).

40

display advertising’s impact on online branded...

Documents