FieldExperimentsinMarketing

DuncanSimester

MITSloanSchoolofManagement

January2015

1|P a g e

FieldExperimentsinMarketingMarketingisadiversefieldthatdrawsfromaricharrayofdisciplinesandabroadassortmentofempiricalandtheoreticalmethods.Oneofthosedisciplinesiseconomicsandoneofthemethodsusedtoinvestigateeconomicquestionsisfieldexperiments.Thehistoryoffieldexperimentsinthemarketingliteratureissurprisinglylong.EarlyexamplesincludeCurhan(1974)andEskinandBaron(1977),whovaryprices,newspaperadvertising,anddisplayvariablesingrocerystores.Thischapterreviewstherecenthistoryoffieldexperimentsinmarketingbyidentifyingpaperspublishedinthelast20years(between1995and2014).Wereporthowthenumberofpaperspublishedhasincreasedduringthisperiod,andevaluatedifferentexplanationsforthisincrease.Wethengroupthepapersintofivetopicsandreviewthepapersbytopic.Thechapterconcludesbyreflectingonthedesignoffieldexperimentsusedinmarketing,andproposingtopicsforfutureresearch

1. PapersthatReportFieldExperiments

Wefocusonthefiveleadingmarketingjournalsthatpublishpaperswithaneconomicsfocus.Theyinclude:theJournalofMarketing(JM),theJournalofMarketingResearch(JMR),MarketingScience(MktSci),QuantitativeMarketingandEconomics(QME),andthemarketingdepartmentofManagementScience(MngSci).1Toidentifyrelevantpaperswefirsthadaresearchassistantreadeveryissueofthejournalspublishedbetween1995and2014.WethensupplementedthisinitiallistwithaWebofSciencetopicsearch.

Thisprocessyieldedover300papers.Wethenreadandscreenedthesepaperstogenerateafinalsampleof61papers.2Inthisscreeningwerestrictedattentiontostudiesinwhichtheresponsemeasurerepresentedabehavioralresponseinthe“field”.Weexcludedstudiesinwhichtheresponsewasasurveyorperceptualmeasure,suchascustomerevaluationsorpurchaseintentions.Wealsoexcludedstudieswheretheresponseswereobtainedinalaboratorysetting,includingstudieswheretheresponseenvironmentwascreatedbytheresearchers.

Thescreeningalsorestrictedattentiontostudiesthatreportedexperiments,ratherthanjustanalysisofhistoricaldata.Anexperimentincludesatleasttwoexperimentalconditions,withexogenousvariationintroducedbytheresearchers.Inmostcasesthisexogenousvariationresultsfromvariationacrossrandomlyselectedcustomersamples.Inothercasesitinvolvesrotationoftreatmentsovertime,ormatchedpairsofproductsinanauctionsetting.

The61papersactuallyreportfindingsfromatotalof86fieldexperiments,with18(30%)ofthepapersreportingmultiplefieldexperiments.Thisincludestwelvepapersthatreporttwoexperiments,fivepapersthatreportthreeexperimentsandonepaperthatreportsfindingsfromfourexperiments.

1ThetwomajormarketingjournalsnotincludedinthislistareJournalofConsumerResearch(JCR)andtheJournalofConsumerPsychology(JCP).Paperspublishedinthesetwojournalshaveastronglypsychologicalperspective.2Althoughwebelievethatthelistof61papersisextensive,itisunlikelytobecomplete.Werecognizethatwewillhaveoverlookedsomepapersthatreportfieldexperiments,andapologizetotheseauthors.

2|P a g e

DateofPublication

Ourfirstinvestigationwastocomparethedatesthepaperswerepublished.ThepublicationdatesaresummarizedinFigure1.Papersreportingfieldexperimentswererarebetween1995and1999,withjustthreeexamplesidentified.Thefrequencyhassinceincreasedsharply;between2010and2014atotalof37fieldexperimentpaperswereidentified.

Figure1:PublicationsbyYear

Thisfigurereportsthenumberofpaperspublishedby5-yearperiod.Thesamplesizeis61(papers).

Therearelikelytobemultiplereasonsforthistrend.Oneexplanationisthatthefieldhasbecomealotmoreconcernedaboutendogeneitywheninterpretingresultsestimatedusinghistoricaldata.Before2000,themarketingliteraturecontainednumerouseconometricpapersstudyinghistoricalsupermarketscannerdata.3Manyoftheseearlypaperspaidlittleattentiontotheendogeneityoftheindependentvariables.Thishassincechanged.Thequantitativemarketingfieldnowpaysconsiderableattentiontothelimitationsinherentininterpretingendogenousvariables.Thefocusonendogeneitycanpartlybeattributedtoa1999paperthathighlightedtheselimitations(Villas-BoasandWiner1999).ThechangealsocoincideswiththedevelopmentofstructuralmodelsinthenewempiricalIOliterature.Publicationinaleadingquantitativemarketingjournalnowrequiresthattheauthorsjustifythesourceoftheiridentification.Manypapersusingeconometricsmethodsnowusestructuralmodelstoaddressthisconcern.Fieldexperimentsprovideresearcherswithanalternativemechanismtoovercomethishurdle.Theincreasedprominenceofconcernaboutendogeneitymayhavecontributedtothesharpincreaseinthenumberofpublishedfieldexperiments.

3ThisliteraturetracesitsoriginstoGuadagniandLittle(1983),whichdemonstratedhowtoapplymultinomiallogittoscannerdata.

3

8

13

37

0

5

10

15

20

25

30

35

40

1995to1999 2000to2004 2005to2009 2010to2014

Num

bero

fPap

ers

3|P a g e

Asecondexplanationforthegrowthinthenumberofpublishedfieldexperimentsisthatithasbecomeeasiertoconductfieldexperiments.ThedevelopmentoftheInternetcoincideswiththegrowthinthenumberofreportedexperiments.ItisnowpossibletoconductfieldexperimentsusingeBayandGoogleAdWords,withoutrequiringactiveparticipationfromacooperatingcompany.Evenforstudiesthatdorequirecooperation,thiscooperationisoftenmucheasiertoobtaininanInternetsettingbecausethecostofconductingtheexperimentsisrelativelylow,andbecausemanyfirmsalreadyconducttheirownexperiments.Firmsthatconductexperimentsaspartoftheirownoperationshaveimplicitlyrevealedthattheyunderstandthevalueoffieldexperiments,andarelikelytohaveinvestedininfrastructuretosupporttheirimplementation.4

Wecanevaluatethisexplanationbyinvestigatingwhethertheformatusedtoimplementexperimentalvariationhaschangedovertime.Inparticular,wegroupedthestudiesintothreecategoriesaccordingtothetypeofexperimentalmanipulation:

Physical in-personinteractionsinahome,ataworkplace,orinabricksandmortarretailstore(includingshelf-signageorproductassortments)

DirectMail telephone,catalogorotherdirectmail

Internet searchordisplayadvertising,eBay,email,Twitter,websitecontent,oranothercomputerorInternetinteraction

InFigure2wereporthowuseofthesemanipulationshaschangedovertime.Thetotalsamplesizeis60observations.5

Inthefiveyearsbetween2010and2014therewerenineteenpaperspublishedthatusedtheInternettoimplementexperimentalvariation,representingmorethanhalfofthefieldexperimentpaperpublishedduringthisperiod.Nineteenpapersinthisfiveyearperiodcontrastswithjustthreepapersinthepreviousfifteenyears.WeconcludethatitislikelytheInternetcontributedtothegrowthinfieldexperimentpapersacrossthedataperiod.

However,itisnotablethatwealsoseegrowthinthenumberofpapersimplementingexperimentsthroughphysicalmanipulationsin-home,attheworkplace,orinphysicalretailstores.Thereweretwelvepapersreportingphysicalmanipulationsbetween2010and2014,comparedtoatotalofjustninepapersinthepreviousfifteenyears.ItseemsthattheeaseofconductingexperimentsontheInternetisnotacompleteexplanationforthegrowthinthenumberofpublishedpapers.

4WemightexpectthatifitiseasiertoconductfieldexperimentsinInternetsettingsthenpapersthatusethissettingaremorelikelytoreportfindingsfrommultipleexperiments.Itturnsoutthatthereverseistrue;papersreportingfindingsfromInternet-basedfieldexperimentsareactuallylesslikelytoreportmultiplestudies.5Infourpapersthedescriptionofthetreatmentwasinsufficienttoallowclassification.Moreover,threepaperseitherreportedmultiplestudiesusingdifferenttypesofvariation,oruseddifferenttypesofvariationacrossdifferenttreatmentswithinthesamestudy.Thesethreepaperswerecodedasappearinginmultiplecategories.

4|P a g e

Figure2:PublicationsbyTypeofExperimentalManipulation

Thisfigurereportsthenumberofpaperspublishedusingeachtypeofexperimentalmanipulationineach5-yearperiod.Thesamplesizeis60.

Athirdexplanationisthatthetopmarketingjournalsarepublishingmorepapers(notjustmorefieldexperimentpapers).In2006MktSciincreasedfromfourissuesayeartosixissuesayear,andin2008JMandJMRmadethesametransition.Inaddition,QMEbeganpublicationin2003.Theresultisanincreaseinthetotalnumberofpublishedpapers.Toinvestigatetheextenttowhichthisexplainstheincreaseinthefrequencyoffieldexperimentpapers,wecountedthetotalnumberofpaperspublishedacrossthe20-yearperiod(1995to2014)andreportahistogramoftheproportionpublishedineach5-yearperiod.Wedothisseparatelyforallpapers,andforjustthosepapersthatreportfieldexperiments.6ThefindingsarereportedinFigure3.

Whiletherehasbeengrowthinthetotalnumberofpaperspublished,thisgrowthisalotsmallerthanthegrowthinthenumberoffieldexperimentpapers.Whencountingallofthepaperspublishedbetween1995and2014,wefindthat31%werepublishedinthelast5years(2010-2014).Incontrast,64%ofallfieldexperimentspaperswerepublishedinthatperiod.

Weconcludethatthesharpgrowthinthenumberofmarketingpapersreportingfieldexperimentscanbeattributedtoseveralfactors.Thefielditselfhasgrownandsotherehasbeengrowthinthenumberofpaperspublished,includingfieldexperimentandnon-fieldexperimentpapers.Second,theeaseofconductingfieldexperimentsovertheInternetmayalsohavecontributedtothephenomenon.Finally,themarketingfieldnowpaysmoreattentiontotheendogeneityofindependentvariables.Fieldexperimentsareaneffectivemechanismforresolvingconfoundsduetoendogeneity.

6ThetotalnumberofpapersinMktSci,JMandJMRarecalculatedusingentriesintheWebofScience.WeexcludedpaperspublishedinMngSciastheWebofSciencedoesnotidentifywhichdepartmenteditoracceptedthepaper.UnfortunatelytheWebofSciencedoesnotindexthefirstfewissuesofQME,andsoforthisjournalwemanuallycountedthenumberofpaperspublishedineachissue.

0

5

10

15

20

1995to1999 2000to2004 2005to2009 2010to2014

Physical DirectMail Internet

5|P a g e

Figure3:HistogramoftheNumberofPapersPublishedinEach5-YearPeriod

AllPapersComparedtoFieldExperimentPapers

Thisfigurereportsahistogramofhowmanypaperswerepublishedineach5-yearperiod.Weseparatelyreportthefindingsforallpapers,andforjustthosepapersthatreportfindingsfromafieldexperiment.Thesamplesizeis3,250forallpapers,and61forthefieldexperimentpapers.Thepercentageswithineachcurveaddto100%.

ChoiceofTopics

Inthefigurebelowwecategorizethe61papersintogeneraltopicareas.Thiscategorizationrevealsthatthepapersaredominatedbytwotopics:pricingandadvertising.Furtherinvestigationalsorevealscleardifferencesinthedistributionoftopicsovertime.Priorto2010,halfofthepapers(12of24)addressedpricingtopics,withonlytwopapersinvestigatingadvertisingissues.However,thisdominancehasreversed,withmorepapersinthelastfiveyearsstudyingadvertisingissues(13)thanpricingissues(11).

Understandably,therecentfocusonadvertisinghasbeendominatedbyresearchquestionsrelatedtoInternetadvertising.ManyofthesepapersstudycharacteristicsofInternetadvertisingthatsimplydidnotexistatthestartofthedatasample.Forexample,dynamicretargetingofadvertising(targetingadvertisementsusingindividualbrowsingbehaviorsonotherwebsites)wasnotpossibleearlierinthedataperiod.Similarly,personalizingadvertisingusinginformationpostedonFacebookpagesisarelativelyrecentadvertisingtechnique.

Inthesectionsthatfollowwebrieflysurveythepapersoneachofthesetopics.Wealsosummarizeallofthepapers,groupedbytopic,inatableasanAppendixtothischapter.

19% 18%

31% 31%

5%13%

21%

61%

0%

20%

40%

60%

80%%ofa

llpa

persacrossa

ll20

years

AllPapers FieldExperimentPapers

6|P a g e

Figure4:ChoiceofTopics

Thisfigurereportsthenumberofpublishedpapersbytopic.Thesamplesizeis61papers.

2. PricingTopics

The23papersinvestigatingpricingissuescanbegroupedintofoursub-topics:

1. Whototargetwithdiscounts?2. Whattypesofdiscountsaremosteffective?3. Priceasasignal4. Multi-parttariffsandotherpricingschemes

Theeffectivenessofdiscountshasbeenafavoriteresearchtopicinthemarketingfield.Thisinpartreflectsboththepervasivenessofdiscounts,andrecognitionthattheycanhavetremendouspositiveandnegativeimpactsonunitsalesandfirmprofits.Forexample,Ailawada,Harlam,CésarandTrounce(2007)studytheunitsalesandprofitimpactofeverydiscountofferedbythenationalpharmacychainCVSin2003.Theyreportthatmorethan50%ofthepromotionsareunprofitablebecausethelossofmarginisnotfullyoffsetbytheincrementalsales.Usingaverylarge-scale13-weekfieldtest,inwhichtheysimplywithheldpromotionsin15productcategoriesin400stores,theydemonstratedthatCVScouldimproveitsprofitsbyover$50million.

WhotoTargetwithDiscounts?

Threefieldexperimentscontributetoourunderstandingofwhoshouldreceivediscounts.Dholakia(2006)showsthatsendinga$5-offdiscountcoupontocustomerswhopreviouslypaidfullpricecanactuallyleadtolessdemand.Thisstudywasconductedusingrandomlyselectedcustomersofalarge

23

15

8

4

11

0

10

20

30

Pricing Advertising ModelValidation

ProductDevelopment

Other

7|P a g e

automobileservicingfirm.7AndersonandSimester(2001a)reportthatofferingextendedpaymenttermstocustomersofaprestigiousjewelrycatalogcanlowerdemand.Theauthorsinterpretthiseffectasanadversequalitysignal,withtheinstallmentbillingsignalingthattheproductsaresuitableforcustomerswhoaremoresensitivetopricethanquality.Inalaterstudywithadifferentcatalogretailer,thesameresearchteamreportthatdeepdiscountshavemorepositivelong-termexternalitieswhentargetedatnewcustomerscomparedtoexistingcustomers(AndersonandSimester2004).Fortheexistingcustomersthelong-termeffectswerenegative;thedeepdiscountsresultedinthesecustomersacceleratingtheirpurchasesandbecomingmorepricesensitive.Incontrast,deeperdiscountsincreasedfuturepurchasesbynewcustomers,apparentlyduetomorefavorableexpectationsaboutfutureprices.Collectivelythesestudiessuggestthatdiscountsaremosteffectivewhentargetedatcustomerswhoare:newer,whohavepaidlowerpricesinthepast,andwhoarerelativelymoreconcernedaboutpricethanquality.

WhichDiscountsareMostEffective?

Fieldexperimentshavebeenusedtoaddressnotjustwhoshouldreceivediscounts,butalsowhatformthediscountsshouldtake.Collectivelytheliteraturenowprovidesarichanswertothisquestion.Intheoldestpaperinoursample,HochandDhar(1996)reportthefindingsfromtwofieldexperimentsconductedat86supermarketstoresintheDominick’sFinerFoodssupermarketchain.8Theyshowedthatcouponsleadtoa35%largerincreaseinunitsalesthananequivalentdiscountintheshelf-price.Moreover,becauseredemptionofcouponsisincomplete,thisleadstoa108%largerincreaseinprofits.Intwolarge-scalefieldexperimentsconductedatthe(apparently)same86supermarkets,Wansink,KentandHoch(1998)illustratetwowaysthatretailerscanincreasethenumberofunitsthatcustomerspurchase.Imposingaquantitylimitof12unitspercustomersledtocustomerspurchasingtwiceasmanyunits(perbuyer)comparedtoaquantitylimitof4units.Framingdiscountsusingmultipleunitpromotions(e.g.2for$1.50)insteadofsingleunitpromotions(75-centseach)alsoincreasedthenumberofunitsthatcustomerspurchased.Bothoftheseeffectsareinterpretedasexamplesofcustomersanchoringonretailer-providedcuestodecidehowmanyunitstopurchase.Inanotherexampleofframing,Chen,Marmorstein,TsirosandRao(2012)showthatdescribingadiscountas“50%morefree”ismoreeffectivethan“33%offtheregularprice”,eventhoughbothareeconomicallyequivalent.Theirstudyismotivatedbyrelatedevidencethatcustomerstendtofocusonthe%magnitudeofthediscount,andneglectthebaseoverwhichthepercentageiscalculated.

RamanathanandDhar(2010)presentfindingssuggestingthattheimpactofdifferenttypesofpromotionsmayvaryaccordingtothecustomer’spsychologicalorientation.CustomersenteringaChicagogrocerystorewereprimedtothinkofeitherenjoyingthemselvesoractingsensibly.Thoseprimedtoenjoythemselvespurchasedmoreitemswhentheyreceivedcouponswithlongerexpirationdates,andwhenthecouponswereframedas“Get$xOff”ratherthan“Save$x”.Incontrast,customers

7RelatedfindingsfromtwofieldexperimentsarereportedbyAndersonandSimester(2010),whichispublishedinaneconomicsjournal.Customersofapublishingcatalogandanapparelcatalogwerelesslikelytoplaceasubsequentorderiftheyreceivedacatalogcontaininglowerpricesthantheyhadrecentlypaidforthesameitem.8ThisisthesameretailerthatprovidedthedataforthenowwidelyusedDominick’sscandatapanel.

8|P a g e

primedtoactsensiblypurchasedmoreitemswhentheyreceivedcouponswithshorterexpirationdatesandwhenthecouponsread“Save$x”.

Therehavebeenseveralstudiesinvestigatingtheimpactofframingdiscountsas“freesamples”or“freegifts”.BawaandShoemaker(2004)usetwofieldexperimentstodisentanglethreeeffectsofofferingfreesamples:demandacceleration(temporalsubstitution),cannibalization,anddemandexpansion.Theyshowthatunlikecoupons,freesamplescangeneratepositivelong-termeffectsthatcanendureforasmuchas12monthsaftertheevent.Inalongitudinalstudyinthefinancialservicessector,HaisleyandLoewenstein(2011)investigatedhowofferingcustomersunexpectedgiftsaffectedtheirsubsequentdepositbalances.Theyfoundthatanincreasingsequenceofgifts($35then$100)hadamorepositiveimpactonsubsequentbalancesthanadecreasingsequence($100then$35).Theydescribedthisasan“aversiontodeterioration”.ShuandGneezy(2010)studyhowexpirationdatesongiftcertificatesaffectredemptionrates.Participantswhoreceivedgiftcertificateswithalongerexpirationdatahadmorefavorableperceptionsofthefirm,butwereactuallylesslikelytoredeemthecertificates.Theyattributethisprofitablecoincidencetoprocrastination,andsupportthisclaimusingafollow-upsurvey.LaranandTsiros(2013)investigatehowuncertaintyaboutwhichfreegiftwillbeofferedinfluencetheeffectivenessofthesepromotions.Ifcustomerswereprimedtothinkabouttheirdecisions,theyrespondedmorefavorablyiftheyknewwhatfreegifttheywouldreceive.Incontrast,iftheywereprimedtomakemoreemotionaldecisionsthentheuncertaingiftwasmoreeffective.

Wealsohighlightthreeparticularlyinnovativestudies.Hui,Inman,HuangandSuher(2013)investigatetheimpactofsendingdiscountsthroughmobilephonetechnology.Theyshowthatsendingcouponsforproductsthatarelocatedfurtherawayfromthecustomers’plannedin-storeshoppingpathsareeffectiveatincreasingunplannedpurchases.TsirosandHardesty(2010)studyhowtomosteffectivelyremovediscounts.Theyshowthatphasingoutadiscountgraduallyismoreeffectiveatraisingunitsalesthanremovingthediscountallatonce.Theyattributethisinparttohighercustomerexpectationsaboutfuturepricelevels.Gaurav,ColeandTobacman(2011)studytheadoptionofrainfallinsuranceinruralIndianvillages.Theinsuranceisarelativelysophisticatedfinancialproduct,andtheyfindthattrainingcustomersabouttheproductsisgenerallymoreeffectivethanofferingotherformsofmarketingpromotions.Theexceptionisa“MoneyBack”guaranteeifthefarmersneverhavecausetoclaimontheinsurance,whichworksalmostaseffectivelyasthetrainingprograms.

PriceasaSignal

Athirdgroupofpricingstudiesfocusesonthesignalingroleofprices.Usingdatafromanapparelcatalog,AndersonandSimester(2001b)highlighttheimportanceofrationinghowoftensalesignsareused.Theymotivatethisstudyasatestofanearlierpaperthatarguesthatsalesignsserveacrediblesignalingrole,enablingpoorlyinformedcustomerstoevaluatewhichpricesarelowrelativelytootherpricesinthemarket(AndersonandSimester1998).Akeyfeatureofthatmodelisthatthesalesignsareself-regulating;anyonesignbecomeslesscrediblethemoreoftenitisused.Inarelatedstudy,AndersonandSimester(2003)measuretheimpactof9-digitpriceendings(e.g.$1.99or$49).Usingaseriesofthreefieldexperimentsconductedattwodifferentwomen’sapparelcatalogs,theyshowthat$9priceendingsincreasedemand,andthatthisdemandincreaseisstrongeronneweritemsthanforestablisheditems.However,theeffectappearstobeweakenedwhenthe$9dollarendingis

9|P a g e

accompaniedbya“sale”sign.Theyinterpretthesefindingsasevidencethatthepriceendingsserveasimilarsignalingroleto“sale”signs,revealingtocustomerswhichitemsarediscountedcomparedtootherpricesinthemarket.

Gneezy,GneezyandLauga(2014)investigatetheinformationthatpricessignalaboutquality.Whencustomersseehighpricestheyanticipatehigherquality,andsoreactmoreadverselyifqualityislow.Asaresult,customersevaluatealowqualityproductwithahighpricemorenegativelythanalowqualityproductwithalowprice.TheyestablishthisresultusingafieldexperimentatasmallCaliforniavineyard.

Twootherpapersinthissignalingstreamhavealreadybeendiscussedascontributionstothediscountingliterature.RecallthatAndersonandSimester’s(2001a)evidencethatextendedpaymenttermsloweredsalesataprestigiousjewelrycatalogwasinterpretedasevidencethattargetingpricesensitivecustomerscanunderminethecredibilityofaretailer’squalityclaim.Similarly,theevidencethatdeeperdiscountstofirst-timecustomerscanmakethesecustomersmoreloyalalsohadasignalinginterpretation(AndersonandSimester2004),althoughinthiscasethesignalwasaboutfuturepricesratherthanquality.

Multi-PartTariffsandOtherPricingSchemes

Thefourthstreamofpricing-relatedfieldexperimentsaddressesmulti-periodandmulti-parttariffstogetherwithotherinnovativepricingschemes.LambrechtandTucker(2012)reportoneofthefewfieldexperimentsconductedinabusiness-to-businesssetting.TheycollaboratewithawebhostingproviderintheUnitedKingdomtoinvestigatetheimpactofvaryingboththemonthlypriceandtheframingof“hasslecosts”inthefirstperiodofamulti-periodcontract.Ratherthanevaluatingtheentirecontractterm,theyshowthatcustomersinthismarketevaluatetheoutcomeineachdistinctperiod.Asaresult,theyshowthatifcustomersincurhasslecostsinthefirstperiod,thenitismoreeffectivetolowerthefirstperiodpriceanddefersomeofthisrevenueuntillaterperiods.

AndersonandSimester(2008)showthatpricedifferentiationcanlowerdemandifitisperceivedtobeunfair.Theirfieldexperimentwasconductedinawomen’sapparelcatalogthatsellsplus-sizes.Becausethecostofproducingverylargegarmentsishigherthanthecostofproducingsmallergarments,theretailersoughttochargehigherpricesonitslargestsizes.Theresultsrevealedanasymmetry;therewasnochangeindemandatthesmallersizes,butdemandfellsharplyforthelargersizes(evenaftercontrollingfortheabsolutepriceitself).Theyinterpretthesefindingsasevidencethatpricedifferentiationcanlowerdemandifitisperceivedtobeunfair.However,customersactuallyhavetoexperiencetheunfairness;itisnotsufficienttoseeothercustomerstreatedunfairly.

Twopapersfocusonmulti-parttariffschemes.Danaher(2002)usestheexogenousvariationintroducedthroughafieldexperimentinthetelecommunicationsmarkettodevelopanoptimalpricingmodelfora2-parttariff.Theschemeincludesbothamonthlyaccessfee,togetherwithaper-minuteusagerateforcellularphoneservice.Theresultshighlighttheimportanceofaccountingforbothusageandcustomerretention.Solelyfocusingonusageandignoringattritionwillsubstantiallyunder-estimatethesensitivityofrevenuetoprice.Yao,Mela,ChiangandChen(2012)alsostudycellularphonepricing,althoughintheircasetheyuseafieldexperimenttorecoverestimatesofcustomerdiscountrates(ratherthan

10|P a g e

revenuepricesensitivity).Customers’planchoicesrevealmuchfasterconsumerdiscountingoffutureperiodsthanwenormallyassume.

Perhapsthemostunusualpricingschemerepresentedintheliteratureisa“pay-what-you-want”(PWYW)scheme.Kim,NatterandSpannreportthreefieldexperimentsconductedinGermanrestaurantsandamovietheatre.ThefirmsrotatedPWYWschemesandtheirstandardpricingschemesondifferentdays.Ingeneral,customerschosetopaylowerpricesthanthepricesthefirmsregularlycharged.However,thiswasnotalwaysthecase.Customersinoneoftheexperimentschosetopayanaverageof$1.94forahotbeverage,comparedtotheregularpriceof$1.75.Inanotherexperiment,customersonaveragechosetopaylessforabuffetlunch($6.44)thantheregularprice($7.99),butthePWYWschemeattractedenoughextrademandtogeneratesignificantlyhigherdailyrevenue.

3. AdvertisingTopics

Thefifteenpapersthatfocusonadvertisingrelatedissuescanbegroupedintothreebroadsub-topics:

1. Doesadvertisingimpactpurchasing?2. Whichadvertisingmessagesaremosteffective?3. Optimizingpaidsearchadvertising

IsthereanImpactonPurchasing?

Fourpapersinvestigatewhetheradvertisingcanimpactpurchasing,includingtwopapersthatreacharelativelynegativeconclusion.LewisandReiley(2014)studywhetherdisplayadvertisementsatYahoo!causedofflinepurchasesatadepartmentstore.Despiteacontrolledrandomizedenvironmentandasamplesizeof1.6millioncustomers,theyareonlyjustabletoestablishastatisticallysignificanteffect.LambrechtandTucker(2013)studytheeffectsof“dynamicretargeting”ofadvertisements.Dynamicretargetingdescribesthewidelyusedpracticeofusingbrowsinghistoryfromotherwebsitestoselectwhichadvertisingcontenttodisplay.Theyfindthatthispracticedoesnotincreaseadvertisingeffectiveness.Theexceptioniswhenthebrowsinghistoryindicatesthatacustomerisrelativelyclosetomakingapurchase.Itisonlythenthatdynamicretargetingiseffective.

Anotherpaperwithakeyfindingthatiseffectivelyanullresultfocusesontheimpactofdigitalvideorecorders(DVRs)onadvertisingeffectiveness.TheintroductionofDVRsledtoconcernthattelevisionadvertisingwouldbecomelesseffectivebecausecustomerscouldfilteroutadvertising.Inthisverylarge-scalescalestudy,theauthorsfoundnoevidencethatthiswasthecase.Insteadtheauthorsconcludedthatfilteringoutadvertisementsoccursrelativelyinfrequently.Incontrasttothetwopreviouspapers,thiscouldbeconsideredarelativelypositivepaperabouttheimpactofadvertising.ItsuggeststhatthewidespreadconsumeradoptionofDVRtechnologywillnotdiminishtheimpactofTVadvertising.

Thefourthpaperinthissequenceinvestigatestheimpactofcompetitiveadvertising.Competingcatalogretailerssharethenamesandaddressesoftheirbestcustomersonareciprocalbasisinordertolowerthecostsofprospectingfornewcustomers.Usingarandomizedfieldexperiment,AndersonandSimester(2013)measurehowallowingcompetitorstotargetyourcustomerswithcompetingcatalogs

11|P a g e

impactsafirm’sownsalestothosecustomers.Althoughwemightexpectanegativeeffectduetosubstitution,theyactuallyfindthatformanycustomersthecompetitors’advertisingincreasedpurchasesfromtheoriginalfirm.Thispositiveeffectwasparticularlystronginproductcategoriesinwhichcustomerslearnproductsizesthatarefirm-specific(suchasfootwear).Thisleadstheauthorstoattributetheirfindingstotheimportanceofproductstandards,customerlearningandswitchingcosts.

WhichMessagesAreMostEffective?

Anadditionalninepapersfocusonimprovingmessagingtoincreaseadvertisingresponserates.Theyincludeseveralpapersthataddresstargetingorpersonalizationofadvertisingmessages.Forexample,Tucker(2014)reportsafieldexperimentconductedonFacebookthatstudieshowcustomersreacttoadvertisementsthatarepersonalizedbasedonacustomer’spostedpersonalinformation(seealsoLambrechtandTucker2013,whichisdiscussedabove).Fortuitously,theexperimentcoincidedwithawidelypublicizedchangeinFacebook’sprivacypolicies,whichgaveusersmorecontrolovertheirpersonalizationsettings.Thefindingsrevealthatgivingcustomerstheoptionofcontrollingtheirpersonalizationsettingsgreatlyimprovedtheperformanceoftargetedadvertisements.Schumann,vonWangenheimandGroene(2014)studyhowtomitigateadversecustomerreactionstotargetedInternetadvertising.Theyshowthatreciprocityappeals(“Ourserviceisfreeofchargetoyou–targetedadvertisinghelpsusfundit”)aremoreeffectivethanrelevanceclaims(“Youwillseemoreinterestingandlessirrelevantadvertisementsinthefuture”).

Threestudiesinvestigatehowmessagingshouldbecustomizedtodifferentcustomersegmentsusingdimensionsotherthanpriorbrowsingbehavior.BergerandSchwarz(2011)studymessagesthatpromptcustomerstorecommendproductstoothercustomers(word-of-mouth).Theyshowthatmessagesthatlinkabrandtoaproductcuearemoreeffectivewhencustomersdonotalreadyhavestrongassociationsbetweenthebrandandthecue.UsingaGoogleAdWordsfieldexperimentconductedinIsrael,Kronrod,GrinsteinandWathieu(2012)demonstratethattheuseofassertivemessages(“YoumustsavetheMediterranean”)aremoreeffectivewhencustomersalreadybelieveinthecause,butarelesseffectiveformoregeneralcauses,forwhichcustomers’preferencesareweaker.

BeyondKronrod,GrinsteinandWathieu(2012),twootherstudiesinvestigatehowtodesignmessagingthatencouragescustomerstoengageinmoreenvironmentallyfriendlybehavior.Inanambitiousstudy,WhiteandSimpson(2013)collaboratewithalargemetropolitancitytoencourageresidentstoleavetheirgrassclippingsontheground,ratherthandisposeofthemthroughthemunicipalwastesystem(whichsendstheclippingstolandfills).Usinghangersplacedonthefrontdoorsofresident’shome,theytestedsixdifferentmessagingconditions,togetherwithaControlcondition.Theythenmeasuredthereductioninwaste.Varyingthetypeofappealafterprimingafocusoneitherindividualorsocialbenefitshadasignificanteffectonresidents’behavior.Spangenberg,Sprott,GrohmanandSmith(2003)reporttwostudies.Inthefirststudytheyinvestigatehowmessagesinoutdooradvertisingmedia(billboards)canincreaserecycling.Inthesecondstudytheyvarythecontentofmessagesinsertedinahealthclub’smonthlynewslettersandbillingstatement,withthegoalofincreasingmembervisitstotheclub.Theyshowthatframinganappealasaself-predictionincreasesitseffectiveness.Specifically,inthefitnessclubstudy,themessage“Fitnessguilt?”wasmoreeffectiveatincreasingmembervisitsthanthealternativetreatment“Workoutat[fitnessclubname]”.

12|P a g e

TwootherexamplesofmessagingtestswerereportedbyTuckerandZhang(2010and2011).Bothofthesestudiesinvestigatehowreportingparticipationinatwo-sidednetworkincreasesparticipationinthenetwork.Thefirstexample(TuckerandZhang2010)isanotherrareexampleofastudyinabusiness-to-businessmarket.Thesettingwasawebsitethatresemblescraigslist.org,andtheoutcomemeasurewasthenumberofsellerswhochosetolistonthesite.Thefieldexperimentrandomizedwhethertodisplaythenumberofbuysand/orsellersand(ifso)thenumberofbuyersand/orsellerstoclaim.Theyfindthatalargenumberofsellerscandeteradditionallistings,unlessonlythenumberofsellersisrevealed(notthenumberofbuyers).Displayingmanybuyerswillattractmoresellersaslongasthenumberofsellersisalsorevealed.Theyconcludethatmarketswithmorecompetitorscanappearmoreattractivetoentrants,astheremustbesufficientdemandtoattractsomanycompetitors(apositivenetworkexternality).Thesecondstudy,TuckerandZhang(2011),usesdatafromawebsitethatlistsweddingservicevendors.Theyrandomizewhetherthewebsiterevealsthepreviousnumberofclicksonavendor.Wemightexpectthatrevealingthisinformationwouldincreasethepopularityofthevendorswiththemostclicks.Instead,theyshowthatthesamelevelofhistoricalclicksresultsinalargerliftforbrandswithanichemarketposition,comparedtoabrandwithamainstreamposition.Customersappeartoinferthatthesebrandsmustprovidehighqualityinordertoovercometheirnarrowreach.

ThefinalpaperinvestigatingtheimpactofadvertisingmessagesincludesapairofstudiesconductedusingmatchedpairsofmusicCDauctionsoneBay.DholakiaandSimonson(2005)studytheimpactofmessagingthatexplicitlyrecommendsthatpotentialbidders“comparethepriceofthisCDwiththepricesofsimilarCDslistednexttothisone”.Theyfindthatthesemessagesleadtomorecautiousbiddingbehavior.Thewinningbidderstendedtosubmitlaterbids,submitfewerbids,andavoidparticipatinginsimultaneousauctions.Theyinterpretthefindingsasevidencethatmakingcompetingoptionsmoresalientmakescustomersmoreriskaversebecauseopportunitycostsbecomemoreprominent.

OptimizingPaidSearchAdvertising

Twopapersbydifferentteamsinvestigatehowtooptimizepaidsearchadvertising.YangandGhose(2010)beginbyusinganhistoricalpaneldatasettoinvestigatehowtheresponsetopaidsearchadvertisingisaffectedbythepresenceoforganic(unpaid)searchresults.Theythentesttherobustnessofthemodelpredictionsusingan8-weekfieldexperimentonGoogle.Theyconfirmedthatwhenpaidsearchispresent,sothatcustomerscanseebothpaidandorganicsearchresults,thecombinedclick-throughrate(CTR)was5.1%highercomparedtowhentherewasnopaidsearchpresent.Theconversionrate(probabilityofapurchase)alsoincreasesby11.7%.

Agarwal,HosanagarandSmith(2011)investigatetheimportanceofpositioninpaidsearchadvertising.Specifically,theyaskwhetherbeingthefirstsearchresult,secondresult,oralowerresult,impactsbothclick-throughrates(CTR)andconversion(purchases).Aswemightexpect,theclosertothefirstposition,thehighertheCTR.However,theoppositeistrueforconversionrates.Appearinglowerintherankofsearchresultscanactuallyincreaseconversionratesbecauseconversionisconditionalonclicking.Asaresult,onlythemostmotivatedcustomersclickonlowerresults,andthesearethecustomersthatare

13|P a g e

morelikelytoconvert.Givenadvertisersgenerallypayforclicksirrespectiveofconversions,itmaybeoptimalformanyadvertiserstopreferlowerpositionsinthesearchorder.

4. Product-RelatedTopics

Weidentifiedsevenpapersthatpresentfieldexperimentsaddressingproduct-relatedtopics.Thisincludesfourstudiesfocusedonmarketresearchandproductdevelopmentissues,andthreepapersfocusedonproductsizes.

MarketResearchandProductDevelopment

Twostudiesinvestigatetherolethatmeremeasurementcanhaveoncustomerpurchasingbehavior.Chandon,MorwitzandReinartz(2005)asked251customersofanonlinegrocerabouttheirpurchaseintentions.Asecondrandomlyselectedsetof140consumersservedasthecontrol(andwerenotaskedanyquestions).Thestudyrevealedthatthecustomerswhoweresurveyedweresignificantlymorelikelytomakeasubsequentpurchase,andweresignificantlymoreprofitablefortheretailer.Twoyearslater,adifferentresearchteamconductedarelatedstudyintheautomotiveservicesindustry(Borle,Dholakia,SinghandWestbrook2007).Atelephonesurveywasadministeredto3,773randomlyselectedcustomers.Subsequentservicevisitsbythesecustomerswerethencomparedwitharandomlyselectedcontrolsampleof1,227customers.Thiscomparisonrevealedthatparticipatinginthesurveyincreasedpurchasesandmadecustomersmoreresponsivetofirmpromotions.

Theothertwostudiesonthistopicusefieldexperimentstovalidatenewmarketresearchmethods.Urban,Hauser,Qualls,Weinberg,BohlmannandChicos(1997)validateamethodforconductingresearchveryearlyintheproductdevelopmentprocess.Forecastsofnewproductsalesaremorevaluabletheearliertheyareavailable.Theauthorsinvestigatethepossibilityofusingmultimediarepresentationsofpotentialnewproductstoprovideearlierandlesscostlyforecasts.Theyconducttwofieldexperiments,whichconfirmthatmultimediacomputerinteractionscanbothrealisticallyportraycustomers’informationsources,andyieldforecaststhatarenotsignificantlydifferentfromtraditionalmethods.Neslin,Novak,BakerandHoffman(2009)proposeamodelthatisdesignedtomaximizeresponseratestoonlinemarketresearchpanels.Themodelisdynamic,optimizingoveradiscretenumberofstudiesinafiniteperiod,wherethisfiniteperiodcanbeextendedusingarollinghorizon.Theycomparetheirmodelagainstthecurrentmanagerialheuristic,andreportsignificantlyhigherresponserates.

ProductSizesandBundling

Consumers’assessmentsofrelativepackagevolumesareoftenbiasedwhenpackageshavedifferentshapes.Krider,RaghubirandKrishna(2001)investigatethisbiasbycomparinghowcustomersreacttoproductpackagesthathavethesamevolumebutdifferentshapes.Inauniversitycafeteriathatsellscreamcheesetocomplementbagels,theauthorscomparedemandfora¾ozroundtubofcreamcheeseanda¾ozsquaretubofthesamecreamcheese.Customersweresignificantlymorelikelytopurchase2tubsofcreamcheesefortheirbagelondaysthatonlytheroundtubsofcreamcheesewereavailable.Thisfindingisconsistentwithotherevidenceintheirpaperthatcustomersperceiveroundcontainerstobesmallerthanrectangularcontainersofthesamevolume.

14|P a g e

LeszczycandHaubl(2010)reportfindingsfromthreefieldteststhatinvestigatetheprofitabilityofproductbundling.UsingaseriesofeBayauctionsofcollectablepostagestamps,theycomparetheprofitabilityofbundlingrelatedandunrelateditems.Theirfindingsrevealthatbundlingsubstitutesorunrelateditemsinanauctionislessprofitablethansellingthemseparately,butbundlingismoreprofitablewhentheitemsarecomplementary.Athirdstudyrelatedtoproductbundlinghasalreadybeendiscussedasanexampleofapricing-relatedpaper.Recallthatinanexperimentconductedat86supermarketstores,Wansink,KentandHoch(1998)comparedtheeffectivenessofmultipleunitpromotions(e.g.2for$1.50)versussingleunitpromotions(75-centseach).Themultipleunitpromotionsincreasedthenumberofunitsthatcustomerspurchasedbyanaverageof32%.

5. ModelValidation

Themarketingfieldhasalongtraditionofdevelopingmodelsthatoptimizemarketingdecisions.Traditionallythesemodelsarevalidatedbymeasuringgoodnessoffit,eitherin-sampleorwithholdoutsamples.Alimitationofthisvalidationisthatitgenerallyreliesonassumptionsinherentinthemodels,andsoerrorsintheassumptionsarealsointroducedtothevalidationprocess.

Becauseoftheselimitations,researchershavebegunusingfieldexperimentsasameansofvalidatingmarketingmodels.Fieldexperimentsprovideanalmostidealvalidationsetting;differentpolicescanbeimplementedintreatmentandcontrolsettingsandtheiroutcomescompared.Thisprovidesa“model-free”basisforvalidation,togetherwithacomprehensivetestofalloftheassumptionsinthemodel.

Forexample,Simester,SunandTsitsiklis(2006)proposeamodelfordynamicallyoptimizingcatalog(andotherdirectmarketing)mailingdecisions.Catalogfirmsregularlysendcatalogstocustomers,andmustdecidewhoshouldreceiveeachcatalog.Traditionallythesedecisionshavebeenmademyopically;thecompaniessendcatalogstocustomerswhoaremostlikelytorespondtothatcatalog.Themodelproposedinthispaperoptimizesasequenceofmailingdecisionsoveraninfinitehorizon.ThemodelbeginsbyproposingamethodfordividingcustomersintodiscreteMarkovstatesusingasetofvariablesdescribingeachcustomer’spurchasingandmailinghistories.Transitionprobabilitiesandrewardsarethenestimatedforeachstrategy(mailornotmail)ineachstatespace.Finally,astandardpolicy-iterationalgorithmisusedtocalculatetheoptimalpolicyineachstate.Theauthorsthentesttheirproposedmethodbyvaryingcatalogmailingdecisionsfor60,000customersofanapparelcatalogoveraperiodofsixmonthsincluding12catalogmailingdates.Thefindingsrevealedthatthemodelperformedwellforlow-valuedandmoderately-valuedcustomers.However,duringtheinitialmonthsoftheexperiment,theresultswerelessfavorableforthemostvaluablecustomersinthesample.Furtherinvestigationrevealedanexplanationforthepoorinitialoutcomeforthesecustomers:inthetrainingdatathereweretoofewoccasionsinwhichthefirmhadnotmailedtothesecustomerstoprovideareliableestimateoftheoutcome.

15|P a g e

Thesecomparisonsnotonlyvalidatetheproposedmodels,buthavealsoprovedvaluableasasourceofinsightsabouttheunderlyingphenomenon.9

InadditiontoSimester,SunandTsitsiklis(2006),weidentifiedsevenotherpaperspublishedinthelasttwentyyearsthatemployfieldexperimentstovalidateoptimizationmodels.Theseincludetwopricingmodels,threeadvertisingmodelsandtwonewmarketresearchmethods.Allbutoneofthesepaperswaspublishedin2006orlater,suggestinggrowinginterestinthisapproach.Althoughsomeofthepapersinvolverelativelysmallnumbersofparticipants(seeforexampleUrbanetal.1997;andBelloni,Lovett,BouldingandStaelin2012),othersincludelargenumbersofparticipantsandextendedtreatmentperiods.Forexample,Mantrala,Seetharaman,Kaul,GopalakrishnaandStam(2006)comparetheoutcomewhenimplementinganoptimalpricingmodelin200experimentalstoresoveraperiodof8weeks.

6. OtherTopics

Oursearchrevealedanadditional10studiesthatdonotfiteasilywithinthepreviousfourtopics.Notably,thesepapersareallrelativelyrecent,withthefirstpublishedin2008,andeightofthempublishedinthelastfiveyears.Wehavegroupedthesepapersintofivetopics:

1. Salesforceoptimization2. Word-of-mouthandreferrals3. Participationinonlinecommunities4. Encouragingpositivebehaviors5. Othertopics

SalesForceOptimization

Kumar,VenkatesanandReinartz(2008)reportfindingsfromtwolarge-scalefieldexperiments.Oneexperimentwasconductedwithamultinationalbusiness-to-businesstechnologyfirm,whilethesecondstudywasconductedwithafirminthetelecommunicationsindustrythatsellstobothbusinessesandretailconsumers.Thestudiescompareda“customer-focus”inwhichthetimingofsalescallswascoordinatedwithforecastsofcustomers’purchasingdecisions.Inthiscondition,salescallswerealsocoordinatedacrossproductcategories,sothatifacustomerwasexpectedtopurchaseinmultiplecategoriesassignedtodifferentsalesteams,thecustomerreceivedajointvisitfrombothteams.Thiscoordinatedpolicywascomparedwithastandardpolicy,whichlackedcoordination.Thetwotreatmentswererandomlyassignedwithinmatchedpairsofcustomers.Thefindingsrevealedthatcoordinationledtosignificantlyhigherprofitsandreturn-on-investment.Lim,AhearneandHam(2009)alsoinvestigatesalesforceoptimizationissuesusingrandomizedfieldexperiments,althoughtheyfocusonsalesforceincentives.Specifically,theycomparetheimpactofdifferentprizestructuresinsalescontests.Theirfindingsindicatethatasalescontestshouldincludemultipleprizewinners,andrankorderingprizesincontestswithmultiplewinnersdoesnotincreasesalesorrevenues.9ThefindingsintheSimester,SunandTsitsiklis(2006)fieldexperimentledtoasubsequentpaper(Mannoretal.2007)inwhichtheauthorsdocumentedthepotentialforpositivebiaswhenapplyingdynamicprogrammingmodelstofielddata.

16|P a g e

Word-of-MouthandReferrals

Inawidely-citedpaper,GodesandMayzlin(2009)investigatewhetherfirmactionsdesignedtoencourageword-of-mouth(WOM)leadtohighersales.TheyfindthatWOMismoreeffectiveatincreasingsalesifitcomesfromrelativelylessloyalcustomers.TheyalsoreportthatWOMgeneratesmoresalesifitcomesfromacquaintances,ratherthanclosefriends.Kumar,PetersonandLeone(2010)usefourfieldexperimentsconductedwithafinancialservicesfirmtoidentifywhichcustomersafirmshouldtargetwhendesigningcustomerreferralprograms.Theyusethefindingstovalidateanapproachforcomputingthe“customerreferralvalue”foreachcustomer.

ParticipationinanOnlineCommunity

InastudyconductedinGermanyinvolvingeBayusers,Algesheimer,Borle,DholakiaandSingh(2010)measuretheimpactofaprogramdesignedtoincreaseparticipationinthefirm’sonlinecommunity.Customersintherandomlyassignedtreatmentgroupreceivedmultipleemailmessagesinvitingparticipation,whilethoseinthecontrolgroupdidnotreceivethesemessages.Overthenextyearcustomersinthetreatmentgroupspentlessandlistedfeweritemsthancustomersinthecontrolgroup.

Twoyearslateranoverlappingresearchteamconductedafollow-upstudyagainusingGermaneBayusers(Zhu,Dholakia,ChenandAlgesheimer2012).Emailmessagestoarandomlyselectedtreatmentgroupwereagainusedtoinviteparticipationinthefirm’sonlinecommunity.Thefindingsrevealthatparticipantsengageinmorerisk-seekingbiddingbehavior.Thiseffectisonlytruewhencommunitymembershavestrongtiestoothermembersofthecommunity.Thefindingsarereplicatedinasecondfieldexperimentconductedwithprosper.com.

ToubiaandStephen(2013)studywhypeoplecontributetothemicrobloggingsiteTwitter(seealsoourearlierdiscussionofBergerandSchwarz2011,whoaskwhycustomerscontributeword-of-mouth).TheyexperimentallymanipulatedthenumberofTwitterfollowers,andcomparedtheirpostingactivitiestoarandomlyassignedcontrolledgroup.Theirfindingssuggestthatmanyuserscontributebecausetheycareabouthowpeopleperceivethem,ratherthanbecausetheyderiveintrinsicutilityfromtheactivity.

EncouragingPositiveBehaviors

Weearlierdescribedtwopapersthatstudyhowadvertisingmessagescanleadtopro-socialbehavior(WhiteandSimpson2013studywastereduction,andSpangenberg,Sprott,GrohmanandSmith2003studyrecyclingbehavior).Twootherpapershavefocusedonencouragingpositivebehaviors.Raju,RajagopalandGilbride(2010)examinehowtoencouragechildrentochoosehealthierfoodoptions.Themosteffectiveinterventionwastoestablisha“friendlyhealthfuleatingcompetitionwithstudentsatthesamegradelevelfromotherparticipatingschools.”Askingthechildrentosignapledgetoeatmorefruitsandvegetables,andprovidingdirectincentivesforhealthychoices(e.g.pencils,stickers,keychains)alsoledtoimprovedeatinghabits,evenupto10weeksaftertheinterventionsended.

SomanandCheema(2011)comparemethodsforimprovingsavingsratesamonglow-incomelaborersinruralIndia.Financialplannersvisitedthe146familiesinthestudyfor15weeks,andgavethemasavingsgoal,whichwasplacedinasealedenvelope.Experimentalvariationsincludedthesizeofthisgoal,whethertheirpicturesoftheirchildrenwereprintedontheenvelope,andwhetherthesavingsgoalwas

17|P a g e

pooledintoasingleenvelopeorsplitbetweentwoenvelopes.Althoughthesizeofthegoaldidnotaffectthesavingsrate,savingsweresignificantlyhigherinthephotographcondition,andwhenthesavingswerepartitionedintomultipleenvelopes.

OtherTopics

Theremainingtwopapersstudydiversetopics.DaggerandDanaher(2014)reportfindingsfromfieldexperimentsdesignedtomeasurehowremodelingastoreimpactsstoredemand.Thefirstofthesestudieswasconductedatan“equipmentretailerandserviceprovider”thatretaineditsoriginalretailspace,whileremodelingnewspaceinthebuildingtoserveasareplacementretailspace.Theresearchersrandomlyrotatedtheretailoperationsbetweenthenewandoldspacesforaperiodofsixweeks.Theyfoundthattheremodeledspaceincreasedsalessignificantlymorefornewcustomersthanforexistingcustomers.Theyreplicatetheirfindingsinasecondstudyconductedinalargedepartmentstore.

HaruvyandLeszczyc(2010)conductedaseriesofexperimentsusingpairsofsimultaneousauctionstomeasuretheimpactofarangeofauctionfeatures,including:auctiondurations,shippingcosts,thelevelofopenreserveprices,andtheuseofsecretreserveprices.Theresultsrevealconsiderablepricedispersionwithintheauction-pairs.Theyalsoconfirmthattheauctionfeatureshadsignificanteffectsonfinalprices,whichtheauthorsattributetotheroleofsearchcosts.

7. DesigningExperimentsandFutureTopics

Weconcludebyreflectingonwhatisrequiredtopublishfieldexperimentsinthemarketingliterature.Wealsodiscusstherangeoftopicsthathavebeenstudied,andidentifytopicsthatremainrelativelyunder-studied.

DesigningExperimentsfortheMarketingLiterature

Fieldexperimentspublishedinmarketingobviouslysharemanyofthesamecharacteristicsasfieldexperimentspublishedineconomicsjournals.However,therearesomedistinguishingfeatures.Publishinganyempiricalstudyinthemarketingliteraturerequiresmorethanmerelydocumentinganeffect.Researchersareexpectedtoalsoshedlightonthemechanismthatcausestheeffect.Forexample,itisnotsufficienttojustshowthatmultipleunitpromotions(e.g.2for$1.50)aremoreeffectivethansingleunitpromotions(75-centseach).Wansink,KentandHoch(1998)werealsoexpectedtoexplainthisresultasanexampleofamoregeneralphenomenon.Theyinterpretedtheirfindingasanexampleof“anchoringandadjustment”.

Thisrequirementcanbeaformidableobstacleforfieldexperiments,asitisoftennotpossibletointerviewcustomers,ortootherwisecollectintermediateprocessmeasuresthatcanrevealunderlyingcauses.Instead,therearefourapproachesthatresearchershavegenerallyusedtoinvestigatethecauseoftheireffects,andtheyoftenusetheseapproachesincombination.

First,manypapersreportinteractionsratherthanjustmaineffects.Forexample,TuckerandZhang(2011)donotjustreportthemaineffectofrevealingpopularityinformationonawebsite.Instead,theycomparetheseeffectsfornicheversusmainstreambrands.Similarly,BergerandSchwarz(2011)

18|P a g e

comparetheeffectofadvertisingmessageonword-of-mouthforcustomerswhohavestrongandweakassociationsbetweenthebrandandthemessage.Forresearchersseekingtopublishfieldexperimentsinmarketing,itisimportanttogivecarefulthoughttowhatinteractionstheycanmeasure,andhowthoseinteractionswillhelpclarifythecauseoftheireffects.Insightfulinteractionsareoftenviewedasthe“clever”elementofastudy.Ideallytheinteractionsareconsistentwithaproposedargument,andaredifficulttoreconcilewithalternativeexplanations.Indeed,thisisthebenefitofreportinginteractions;theyaremorelikelytobeimmunefromalternativeexplanations.

Toestimateinteractions,researcherssometimesusemultiplefieldexperiments(recallthat30%ofthepapersreportresultsfrommultiplefieldexperiments).Althoughtheadditionalstudiesareoccasionallypositionedasreplications(e.g.Danaher2002),theyarealsooftenusedtoinvestigateinteractions(e.g.AndersonandSimester2003).Otherstudiesreportasingleexperiment,butincludealargenumberofexperimentaltreatments.Forexample,inGaurav,ColeandTobacman’s(2011)studyofdemandforrainfallinsuranceamongruralIndianfarmers,theyinclude14differentexperimentalconditions.Ingeneral,studieseitherreportmultipleexperimentsormultipletreatments(andnotboth).10

Whereapaperreportsasinglefieldexperimentwithjusttwoconditions,itiscommontocomplementthefieldexperimentwithoneormorelaboratoryexperiments.Examplesinclude,Kronrod,GrinsteinandWathieu(2012),whoaddtwolaboratoryexperiments,andKrider,RaghubirandKrishna,whoreportfivelaboratoryexperiments.Inpapersthatincludemultiplelaboratoryexperiments,thefieldexperimentoftenservesalessprominentrole.Inparticular,thelaboratoryexperimentsmayestablishthemaineffect,replicatetheeffect,andinvestigateinteractions,whereasthefieldexperimentisrelegatedtoconfirminggeneralizabilityinafieldsetting.

Athirdapproachusedtoinvestigateexplanationsistocombinethefieldexperimentwithacustomersurvey.Forexample,recallthatAndersonandSimester(2001a)measuredhowcustomersreacttoaninstallmentbillingofferinapremiumjewelrycatalog.Customersdidnotpurchaseasmuchintheinstallmentbillingcondition,whichtheyinterpretedasanadversequalitysignal,revealingthattheproductsaretargetedatcustomerswhoaremoresensitivetoqualitythanprice.Tosupportthisinterpretation,theymailedcatalogssimilartothoseusedfortheexperimentaltreatmentstootherrandomlyselectedcustomersamples,togetherwithashortsurvey.Customerswhoreceivedtheinstallmentbillingversionweremorelikelytoexpressconcernaboutproductquality.Onerespondentwroteonthesurveyinstrument:“Itmakes[catalogname]looktackytohaveinstallmentplans–kindoflikeFranklinMintdolls”(atpage326).ShuandGneezy(2010)provideasimilarexampleofusingasurveytovalidateanexplanationfortheirexperimentalfindings.

Thefourthapproachistousefieldexperimentdatatoestimateastructuralmodel.Forexample,inarecentworkingpaperDubé,LuoandFang(2016)reportfindingsfromtwofieldexperimentsinwhichtheysentSMS(text)messagestoChinesemobilephoneusersofferingpromotionsonmovietickets.Theyrandomlyvaried(a)thesizeofthediscount,(b)whetherthetextindicatedtheserviceproviderwoulddonatetoalocalcharityforeveryticketpurchased,and(c)thesizeofthedonation(ifany).Surprisingly,theyshowthatcustomersarelessresponsivetolargerdonationswhenthediscountsare10Thepair-wisecorrelationbetweenthenumberofexperimentsandthe(maximum)numberofconditionsis−0.14.

19|P a g e

large.Theyestimateastructuralmodeltoprovideevidencethatthiseffectresultsfrom“self-signaling”;largerpricediscountspromptconsumerstoinferthattheirpurchasesarenolongeraltruistic.Theauthorsobservethatwithoutthestructuralmodelthereisnoobviouswaytotestthismechanismanddisentangleitfromalternativeexplanations.Thisapproachofusingastructuralmodeltoexplainthefindingsinafieldexperimentisstillrelativelynovelinthemarketingliterature.Thestructuralmodelingandfieldexperimentliteratureshavegrownatsimilartimes,perhapspartlybecausetheyofferalternativesolutionstoresolvingendogeneityconcerns(seeearlierdiscussion).Thispaperrecognizesthatthetwoapproachesarenotjustalternatives;theycanalsocomplementeachother.Giventherapidgrowthininterestinbothmethods,oncethebenefitsofcombiningthemethodsaremorewidelyrecognizedweshouldanticipatemanymorepapersusingthiscombinationofmethods.

Randomization

Randomizationofferswell-documentedstatisticaladvantageswhencomparingtreatmentandcontrolconditions.However,randomizationisnotrequiredtopublishfieldexperimentsinthemarketingliterature.In29%ofthepapers,experimentaltreatmentswerenotassignedbyrandomization(inthreepapersitwasunclearwhetherthestudiesusedrandomization).Insomecasesrandomizationcouldhaveledtocontagionbetweentheexperimentaltreatments.Forexample,inSomanandCheema’s(2011)studyofhowtoincreasesavingslevelsamongruralIndianlaborers,participantswereassignedtoconditions“accordingtogeographicandsocialclusterstominimizethepossibilityofhouseholdsfromdifferenttreatmentconditionsmeetinganddiscussingtheirparticipation”(atpageS17).Inothercasesitisnotobvioushowrandomizationcouldbeachieved.Forexample,inthestudiesinvolvingmatchedpairsofauctionsoneBay(DholakiaandSimonson2005,LeszczycandHaubl2010,andHaruvyandLeszczyc2011),thematchedpairsserveastreatmentandcontrolsamples.Randomizingtheexperimentaltreatmentsbetweenproductswithinapairwouldnotbemeaningful,eitherbecausetheproductsareidentical,orbecausetheexperimentalvariationinvolvesdifferencesintheproductbundlesthemselves.

Intheabsenceofrandomization,acommonapproachistorotatetreatmentsacrosstime.Forexample,inDaggerandDanaher’s(2014)studyofastoreremodeling,“theoriginalretailenvironmentservedasthecontrol,usedinweeks1,4and5.Thenewenvironment,whichrepresentedthetreatmentcondition,wasinplaceforweeks2,3and6”(page66).11OtherexamplesincludeYangandGhose(2010),wherethetreatmentwasimplementedbyrotatingsponsorshipofkeywordsintwoweekintervals,andKrider,RaghubirandKrishna(2001),whorotateacrossdaystheshapeofcreamcheesepackagesinastore.

FutureTopics

Thissurveyoftherecentmarketingliteraturerevealsaremarkablediversityinfieldexperimenttopics.Thismirrorsthediversityoftopicsinthefieldgenerally.Firmshaveawiderangeofleversthattheycanusetoinfluencedemand,andthemarketingliteratureconsequentlyhasawiderangeoftopicsto

11DaggerandDanaher(2014)didrandomlyassignthetreatmentstothesixweeks,althoughitiscommonforrotationtooccurwithoutrandomization.

20|P a g e

choosefrom.Notably,theInternethasincreasedtherangeofleversavailabletofirms,andthishasfurtherbroadenedthetopicsthatarestudied.

Whilethediversityofavailabletopicsmakesthefieldattractivetoresearchers,itisalsoaweakness.Almostwithoutexceptionthepaperswehavediscussedraisenewresearchquestions,ratherthanbuildingonpreviousresearch.Otherfieldsbenefitfromgreaterfocusintheirresearchtopics;suchasthestudyofpricerigidityinthemonetaryeconomicsliterature.Thisisnotthecaseinmarketing,whichlacksbroadagreementabouttheresearchquestionsthatshouldbeprioritized.Asaresult,therehasnotbeenatraditionofextendingpreviousfindingsandcomprehensivelyansweringwell-definedproblems.Althoughtherearesomeexceptionstothisobservation,manyoftheexceptionsreflectadedicationtoaresearchquestionbyasingleresearchteam(orbyteamswithoverlappingresearchers).

Despitethediversityoftopics,therearetopicsthatareclearlyunder-represented.Noneofthepapersinvestigateissuesrelatedtomanagingupstreamordownstreamchannelpartnerships.Forexample,noneofthepapersinvestigatethebenefitsofexclusivityinsupplyordistributionrelationships,ortheroleofstandardizationtofacilitatecoordination.Theabsenceofstudiesonthesetopicsmayinpartreflectthedifficultyofimplementingexperimentalvariation.Variationinchannelrelationshipsalmostalwaysrequiresconflict,whichdisruptsrelationshipsthatfirmshaveoftenspentconsiderabletimebuilding.

Thereisalsorelativelylittleresearchstudyingbusiness-to-businessmarkets.Onlyfiveofthepapersreportfieldexperimentsinwhichfirmstargetotherbusinesses(thisincludesonestudytargetingruralfarmersinIndia).Thispaucityofstudiesmayreflectdifficultiesingeneratingsufficientsampleinthesemarkets.Experimentationmayalsobehinderedbygreatertransparency,whichcouldleadtobusinesscustomersinthedifferentexperimentalconditionsbecomingawareoftheexperimentalvariation(althoughthislimitationwouldalsoseemrelevantinseveraloftheInternetstudies).Athirdpossibilityisthatthetimerequiredtoimplementchangeandobserveoutcomescouldbelongerinbusiness-to-businesssettingsthaninconsumermarkets.

Whatisperhapsmostsurprisingisthelackofpapersinvestigatingthefeasibilityofusingfieldexperimentstooptimizemarketingdecisions.Aswehavealreadyrecognized,themarketingfieldhasalonghistoryofdevelopingmodelstooptimizemarketingdecisions.Indeed,eightofthepapersthatwereviewedreportonfieldexperimentsdesignedtovalidateexampleofoptimizationmodels.However,fieldexperimentsarethemselvesanoptimizationmethod.Byexperimentallychangingmarketingvariablesandcomparingtheoutcomes,firmscouldinprincipleusefieldexperimentstoimproveprofitsbysearchingacrossthespaceofpossibledecisions.

Noneoftheempiricalpaperswesurveyedinvestigatethefeasibilityofthisapproach.PerhapstheclosestexampleisDanaher(2002),whichusesthedatafromafieldexperimentimplementedbyatelecommunicationscompanytodevelopastatisticalmodelthatoptimizesatwo-parttariff(pricing)scheme.However,inthisexamplethefieldexperimentisusedasasourceofdata,whiletheoptimizationispursuedusingmoretraditionalmethods.

Althoughwecouldfindnoempiricalpapersthatstudywhetherfirmscouldusefieldexperimentsasapracticaloptimizationmethod,thereisarecenttheoreticalpaper.Li,Rusmevichientong,Simester,

21|P a g e

TsitsiklisandZoumpoulis(2015)investigatehowmanyexperimentsareneededtosetpricesinaproductcategory,asthesizeofthecategorygrows.Settingpricesacrossaproductcategoryrequiresestimatingalargematrixofcross-productdemandelasticities(becauseitemsinthesamecategorymaybesubstitutesorcomplements).Theyshowthatifthecategoryhasafavorablestructure,thenumberofexperimentsneededmaygrowjustlogarithmicallywiththenumberofproducts.Theyconcludethatfirmsmaybeabletoobtainmeaningfulestimatesusingapracticallyfeasiblenumberofexperiments,evenincategorieswithalargenumberofproducts.Toourknowledge,thisistheonlypaperthatformallyinvestigateswhenitisfeasibletooptimizemarketingvariablesusingfieldexperimentsalone.However,thispaperpresentstheoreticalresultsusinginformationtheory;itdoesnotreporttheresultsofanyfieldexperiments.

8. Conclusions

Fieldexperimentsarenolongerararityinthemarketingliterature.Thesharpgrowthinthenumberofpapers,particularlyinthelastfiveyears,meansthattheyarenowamainstreammethodforundertakingempiricalresearch.Thegrowthinthenumberofpapersismatchedbytremendousgrowthinthenumberofauthorswhoareparticipatinginthesestudies.Between1995and1999just11authorsauthoredorco-authoredapaperusingafieldexperiment(inoursample).Between2010and2014therewere75authorsrepresented.12

Thereappeartobeatleastthreereasonsforthisgrowthinthenumberoffieldexperimentpapers.First,themarketingfieldhasbecomeincreasinglyconcernedabouttheinterpretationofendogenousindependentvariables.Thisconcerndoesnotariseifexperimentaltreatmentsareexogenouslymanipulated.Second,thefieldhasgrowngenerally,andsotherearemorepapersofalltypesbeingpublished.Third,theInternethasmadeiteasiertoimplementfieldexperiments,oftenwithouttheneedforcooperationfromfirms.

Oursurveyrevealsthatbefore2010,fieldexperimentpapersinmarketingweredominatedbypricingtopics.Since2010,atleastsomeofthisfocushasshiftedfrompricingtoadvertising,withmanyoftherecentpapersinvestigatingtopicsrelatedtoInternetadvertising.However,evenwithinthepricingandadvertisingtopics,thereisaremarkablelevelofdiversityintheresearchquestions.Veryfewofthepapersbuildonapreviousfieldexperimentpaper.Werecognizethisasbothastrengthandweaknessofthefield.

Animportantcriterionforpublishingafieldexperimentinatopmarketingjournalistheabilitytoextendthefindingsbeyonddocumentingamaineffect,toalsoprovideinsightsaboutthecauseofthateffect.Althoughexplainingeffectsisobviouslyvaluedineconomicsjournals,thisobjectivereceivesevengreateremphasisinthemarketingfield.Fieldexperimentsarenotalwayswell-suitedtoevaluatingcompetingexplanations.Forthisreason,manyofthepapersreportinteractions,ratherthanjustmaineffects.Otherresearcherscomplementtheirfieldexperimentswithlaboratoryexperimentsorcustomersurveys,inordertoprovidesupportfortheirinterpretations.

12Acrossall20yearsthereare123uniqueauthorsorco-authorsofpapersinthesample.

22|P a g e

Althoughanexplanationforaneffectisgenerallynecessary,randomizationisnot.Almost30%ofthefieldexperimentspublishedinmarketingdonotemployrandomizationtoassignexperimentaltreatments.Whentreatmentsarenotrandomized,theyaregenerallyrotatedovertime.WealsoidentifiedthreepapersthatreportfieldexperimentsusingmatchedsetsofproductsauctionedoneBay.

Althoughwehavehighlightedthediversityintopics,ourreviewalsoidentifiedseveraltopicsthatareyettoreceiveattention.Theseincludetheproblemsofmotivatingandcoordinatingupstreamanddownstreamchannelrelationships.Therearealsoveryfewstudiesofbusiness-to-businessmarkets.Finally,werecognizedthatfieldexperimentsarethemselvesanoptimizationmethod;firmscouldusefieldexperimentstoimproveprofitsbysearchingacrossthespaceofpossibledecisions.Althoughthereissomeinitialtheoreticalworkinthisarea,thefeasibilityofusingfieldexperimentsinthisroleremainsanimportantbutunder-studiedtopic.

23|P a g e

9. References

Agarwal,Ashish,KartikHosanagar,andMichaelD.Smith(2011),“Location,Location,Location:AnAnalysisofProfitabilityofpositioninonlineAdvertisingMarkets,”JournalofMarketingResearch,Vol.XLVIII,December,1057-1073.

Ailawadi,KusmL.,BariA.Harlam,JacquesCésar,andDavidTrounce(2007),“QuantifyingandImprovingPromotionEffectivenessatCVS,”MarketingScience,26(4),566-575.

Algesheimer,René,SharadBorle,UtpalM.Dholakia,andSiddarthS.Singh(2010),“TheImpactofCustomerCommunityParticipationonCustomerBehaviors:AnEmpiricalInvestigation,”MarketingScience,29(4),756-769.

Anderson,EricT.andDuncanI.Simester(1998),“TheRoleOfSaleSigns”MarketingScience,17(2),139-155.

Anderson,EricT.andDuncanI.Simester(2001a),“ResearchNote:PriceDiscriminationasaSignal:WhyanOffertoSpreadPaymentsMayHurtDemand,”MarketingScience,20(3),315-327.

Anderson,EricT.andDuncanI.Simester(2001b),“AreSaleSignsLessEffectiveWhenMoreProductsHaveThem?”MarketingScience,20(2),121-142.

Anderson,EricT.andDuncanI.Simester(2003),“Effectsof$9PriceEndingsonRetailSales:EvidencefromFieldExperiments,”QuantitativeMarketingandEconomics,1(1),93-110.

Anderson,EricT.andDuncanI.Simester(2004),“LongRunEffectsofPromotionDepthonNewVersusEstablishedCustomers:ThreeFieldStudies,”MarketingScience,23(1),4-20.

Anderson,EricT.andDuncanI.Simester(2008),“ResearchNote:DoesDemandFallWhenCustomersPerceiveThatPricesAreUnfair?TheCaseofPremiumPricingforLargeSizes,”MarketingScience,27(3),492-500.

Anderson,EricT.andDuncanI.Simester(2010),“PriceStickinessandCustomerAntagonism,”QuarterlyJournalofEconomics,125(2),729-765.

Anderson,EricT.andDuncanI.Simester(2013),“AdvertisinginaCompetitiveMarket:TheRoleofProductStandards,CustomerLearningandSwitchingCosts,”JournalofMarketingResearch,50(4),489-504.

Bawa,Kapil,andRobertShoemaker(2004),“TheEffectsofFreeSamplePromotionsonIncrementalBrandSales,”MarketingScience,23(3),345-363.

Belloni,Alexandre,MitchellJ.Lovett,WilliamBoulding,andRichardStaelin(2012),“OptimalAdmissionandScholarshipDecisions;ChoosingCustomizedMarketingOfferstoAttractaDesirableMixofCustomers,”MarketingScience,31(4),621-636.

Berger,JonahandEricM.Schwarz(2011),“WhatDrivesimmediateandOngoingWordofMouth,”JournalofMarketingResearch,Vol.XLVIII,October,869-880.

Borle,Sharad,UtpalM.Dholakia,SiddharthS.Singh,andRobertA.Westbrook(2007),“TheImpactofSurveyParticipationonSubsequentCustomerBehavior:AnEmpiricalinvestigation,”MarketingScience,26(5),711-726.

Bronnenberg,BartJ.,Jean-PierreDubé,andCarlF.Mela(2010),“DoDigitalVideoRecordersInfluenceSales?”JournalofMarketingResearch,Vol.XLVII,998-1010.

Chandon,Pierre,VickiG.Morwitz,andWernerJ.Reinartz(2005),“DoIntentionsReallyPredict

24|P a g e

Behavior?Self-GeneratedValidityEffectsinSurveyResearch,”JournalofMarketing,69,April,1-14.

Chen,Haipeng(allan),HowardMarmorstein,MichaelTsiros,andAkshayR.Rao(2012),“WhenMoreisLess:TheImpactofBaseValueNeglectonConsumerPreferencesforBonusPacksoverPriceDiscounts,”JournalofMarketing,76,July,64-77.

Curhan,RonaldC.(1974),“TheEffectsofMerchandisingandTemporaryPromotionalActivitiesontheSalesofFreshFruitsandVegetablesinSupermarkets,”JournalofMarketingResearch,Vol.XI,286-94.

Dagger,TraceyS.,andPeterJ.Danaher(2014),“ComparingtheEffectofStoreRemodelingonNewandExistingCustomers,”JournalofMarketing,78,May,62-80.

Danaher,PeterJ.(2002),“OptimalPricingofNewSubscriptionServices:AnalysisofaMarketExperiment,”MarketingScience,21(2),119-138.

Dhar,Sanjay,andStephenJ.Hoch(1996),“PriceDiscriminationUsingIn-StoreMerchandising,”JournalofMarketing,January,17-30.

Dholakia,UtpalM.(2006),“HowCustomerSelf-DeterminationInfluencesRelationalMarketingOutcomes:EvidencefromLongitudinalFieldStudies,”JournalofMarketingResearch,Vol.XLIII,February,109-120.

Dholakia,UtpalM.,andItamarSimonson(2005),“TheEffectofExplicitReferencePointsonConsumerChoiceandOnlineBiddingBehavior,”MarketingScience,24(2),206-217.

Dubé,Jean-Pierre,XuemingLuo,andZhengFang(2016),“Self-SignalingandProsocialBehavior:aCauseMarketingExperiment,”workingpaper,UniversityofChicago.

Eskin,GeraldJ.andPennyH.Baron(1977),“EffectsofPriceandAdvertisinginTest-MarketExperiments,”JournalofMarketingResearch,Vol.XIV,499-508.

Gaurav,Sarthak,ShawnCole,andJeremyTobacman(2011),“MarketingComplexFinancialProductsinEmergingMarkets:EvidencefromRainfallInsuranceinIndia,”JournalofMarketingResearch,Vol.XLVIII,SpecialIssue2011,S150-S162.

Gneezy,Ayelet,UriGneezy,andDominiqueOliéLauga(2014),“AReference-DependentModelofthePrice-QualityHeuristic,”JournalofMarketingResearch,Vol.LI,April,153-164.

Godes,DavidandDinaMayzlin(2009),“Firm-Created-Word-of-MouthCommunication:EvidencefromaFieldTest,”MarketingScience,28(4),721-739.

Guadagni,PeterM.andJohnD.C.Little(2008),“ALogitModelofbrandChoiceCalibratedonScannerData,”MarketingScience,15(1),1-20.

Haisley,EmilyandGeorgeLoewenstein(2011),“It’sNotWhatYouGetbutWhenYouGetIt:TheEffectofGiftSequenceonDepositBalancesandCustomerSentimentinaCommercialBank,”JournalofMarketingResearch,Vol.XLVIII,February,103-115.

Haruvy,Ernan,andPeterT.PopkowskiLeszczyc(2010),“SearchandChoiceinOnlineConsumerAuctions,”MarketingScience,29(6),1152-1164.

Hui,SamK.,J.JeffreyInman,YanliuHuang,andJacobSuher(2013),“TheEffectofIn-StoreTravelDistanceonUnplannedSpending:ApplicationstoMobilePromotionStrategies,”JournalofMarketing,77,March,1-16.

Ju-YoungKim,MartinNatterandMartniSpann(2009),“PayWhatYouwant:ANewParticipativePricingMechanism,”JournalofMarketing,73,January,44-58.

25|P a g e

Krider,RobertE.,PriyaRaghubir,AradhnaKrishna(2001),“Pizzas:πorSquare?PsychologicalBiasesinAreaComparisons,”MarketingScience,20(4),405-425.

Kronrod,Ann,AmirGrinstein,andLucWathieu(2012),“GoGreen!ShouldEnvironmentalMessagesBeSoAssertive?”JournalofMarketing,76,95-102.

Kumar,V.,J.AndrewPeterson,andRobertP.Leone(2010),“DrivingProfitabilitybyEncouragingCustomerReferrals:Who,When,andHow,”JournalofMarketing,74,September,1-17.

Kumar,V.,RajkumarVenkatesan,andWernerReinartz(2008),“PerformanceImplicationsofAdoptingaCustomer-FocusedSalesCampaign,”JournalofMarketing,72,September,50-68.

Lambrecht,AnjaandCatherineTucker(2012),“PayingwithMoneyorEffort:PricingWhenCustomersAnticipateHassle,”JournalofMarketingResearch,Vol.XLIX,66-82.

Lambrecht,AnjaandCatherineTucker(2013),“WhenDoesRetargetingWork?InformationSpecificityinOnlineAdvertising,”JournalofMarketingResearch,Vol.L,October,561-576.

Laran,Juliano,andMichaelTsiros(2013),“AnInvestigationoftheEffectivenessofUncertaintyinMarketingPromotionsInvolvingFreeGifts,”JournalofMarketing,77,112-123.

Leszczyc,PeterT.L.Popkowski,andGeraldHaübl(2010),“ToBundleorNottoBundle:DeterminantsoftheProfitabilityofMulti-ItemAuctions,”JournalofMarketing,Vol.74,110-124.

Lewis,RandallA.,andDavidH.Reiley(2014),“OnlineAdsandOfflineSales:MeasuringtheEffectsofRetailAdvertisingViaaControlledExperiment,”QuantitativeMarketingandEconomics,12,235-266.

Li,Hongshuang(Alice),andP.K.Kannan(2014),“AttributingConversionsinaMultichannelOnlineMarketingEnvironment:AnEmpiricalModelandaFieldExperiment,”JournalofMarketingResearch,Vol.LI,February,40-56.

Li,JimmyQ.,PaatRusmevichientong,DuncanI.Simester,JohnN.TsitsiklisandSpyrosI.Zoumpoulis(2015),“TheValueofFieldExperiments,”ManagementScience,forthcoming.

Lim,Noah,MichaelJ.Ahearne,andSungH.Ham(2009),“DesigningSalesContests:DoesthePrizeStructureMatter?,”JournalofMarketingResearch,Vol.XLVI,356-371.

Mantrala,MuraliK.,P.B.Seetharaman,RajeeveKaul,andSrinathGopalakrishna,andAntonieStam(2006),“OptimalPricingStrategiesforanAutomotiveAftermarketRetailer,”JournalofMarketingResearch,Vol.XLIII,November,588-604.

Neslin,ScottA.,ThomasP.Novak,KennethR.Baker,andDonnaL.Hoffman(2009),“AnOptimalContactModelforMaximizingOnlinePanelResponseRates,”ManagementScience,55(5),727-737.

Raju,Sekar,PriyaliRajagopal,andTimothyJ.Gilbride(2010),“MarketingHealthfulEatingtoChildren:TheEffectivenessofIncentives,PledgesandCompetitions,”JournalofMarketing,Vol.74,May,93-106.

Ramanathan,SureshandSanjayK.Dhar(2010),“TheEffectofSalesPromotionsontheSizeandCompositionoftheShoppingBasket:RegulatoryCompatibilityfromFramingandTemporalRestrictions,”JournalofMarketingResearch,Vol.XLVII,June,542-552.

Schumann,JanH.,FlorianvonWangenheim,andNicoleGroene(2014),“TargetedOnlineAdvertising:UsingReciprocityAppealstoIncreaseAcceptanceAmongUsersofFreeWebServices,”JournalofMarketing,78,January,59-75.

Shu,SuzanneB.andAyeletGneezy(2010),“ProcrastinationofEnjoyableExperiences,”JournalofMarketingResearch,Vol.XLVII,October,933-944.

26|P a g e

Simester,DuncanI.,PengSun,andJohnN.Tsitsiklis(2006),“DynamicCatalogMailingPolicies,”ManagementScience,52(5),683-696.

Skiera,Bernd,andNadiaAbouNabout(2013),“PROSAD:ABiddingDecisionSupportSystemforProfitOptimizingSearchEngineAdvertising,”MarketingScience,32(2),213-220.

Soman,Dilip,andAmarCheema(2011),“EarmarkingandPartitioning:IncreasingSavingbyLow-IncomeHouseholds,”JournalofMarketingResearch,Vol.XLVIII,SpecialIssue2011,S14-S22.

Spangenberg,EricR.,DavidE.Sprott,BiancaGrohmann,andRonnJ.Smith(2003),“Mass-CommunicatedPredictionRequests:PracticalApplicationandaCognitiveDissonanceExplanationforSelf-Prophecy,”JournalofMarketing,Vol.67,July,47-62.

Toubia,Olivier,andAndrewT.Stephen(2013),“Intrinsicvs.Image-RelatedUtilityinSocialMedia:WhyDoPeopleContributeContenttoTwitter?,”MarketingScience,32(3),368-392.

Tsiros,Michael,andDavidM.Hardesty(2010),“EndingaPricePromotion:RetractingItinOneSteporPhasingItOutGradually,”JournalofMarketing,74,January,49-64.

Tucker,CatherineE.(2014),“SocialNetworks,PersonalizedAdvertising,andPrivacyControls,”JournalofMarketingResearch,Vol.LI,October,546-562.

Tucker,CatherineE.,andJuanjuanZhang(2010),“GrowingTwo-SidedNetworksbyAdvertisingtheUserBase:AFieldExperiment,”MarketingScience,29(5),805-814.

Tucker,CatherineE.,andJuanjuanZhang(2011),“HowDoesPopularityInformationAffectChoices?AFieldExperiment,”ManagementScience,57(5),828-842.

Urban,GlenL.,JohnR.Hauser,WilliamJ.Qualls,BruceD.Weinberg.JonathanD.Bohlmann,andRobertaA.Chicos(1997),“InformationAcceleration:ValidationsandLessonsfromtheField,”JournalofMarketingResearch,Vol.XXXIV,February,143-153.

Urban,GlenL.,Guilherme(Gui)Liberali,ErinMacDonald,RobertBordley,andJohnR.Hauser(2014),“MorphingBannerAdvertising,”MarketingScience,33(1),27-46.

Villas-Boas,MiguelJ.,andRussellS.Winer(1999),“EndogeneityinBrandChoiceModels,”ManagementScience,45(10),1324-1338.

Wansink,Brian,RobertJ.Kent,andStephenJ.Hoch(1998),“AnAnchoringandAdjustmentModelofPurchaseQuantityDecisions,”JournalofMarketingResearch,Vol.XXXV,February,71-81.

White,Katherine,andBonnieSimpson(2013),“WhenDo(andDon’t)NormativeAppealsInfluenceSustainableConsumerBehaviors?”JournalofMarketing,77,March,78-95.

Yang,Sha,andAnindyaGhose(2010),“AnalyzingtheRelationshipBetweenOrganicandSponsoredSearchAdvertising:Positive,NegativeorZeroIndependence?”MarketingScience,29(4),602-623.

Yao,Song,CarlF.Mela,JeongwenChiang,andYuxinChen(2012),“DeterminingConsumers’DiscountRateswithFieldStudies,”JournalofMarketingResearch,Vol.XLIX,December,822-841.

Zhu,Rui(Juliet),UtpalM.Dholakia,Xinlei(Jack)Chen,andRenéAlgesheimer(2012),“DoesOnlineCommunityParticipationFosterRiskyFinancialBehavior?”JournalofMarketingResearch,Vol.XLIX,June,394-407.

27|P a g e

Appendix:SummaryofthePapers

PricingIssues

Whototargetwithdiscounts?

Avoidsendingdiscountstocustomerswhopreviouslypaidfullprice. Dholakia(2006)

Avoidofferingdiscountstocustomerswhoaremoresensitivetoqualitythanprice. AndersonandSimester(2001a)

Deepdiscountshaveamorefavorablelong-termimpactfornewcustomersthanforexistingcustomers.

AndersonandSimester(2004)

Whattypesofdiscountsaremosteffective?

Morethan50%ofthepromotionsareunprofitablebecausethelossofmarginisnotfullyoffsetbytheincrementalsales.

Ailawada,Harlam,CésarandTrounce(2007)

Couponsaremoreeffectiveforincreasingsalesandprofitsthandiscountsofftheregularprice.

DharandHoch (1996)

Multipleunitpromotions(e.g.2for$1.50)aremoreeffectivethansingleunitpromotions(75-centseach).Largerquantitylimitsincreasethenumberofunitsthatcustomersbuy.

Wansink,KentandHoch(1998)

Framingadiscountas“50%morefree”ismoreeffectivethan“33%offtheregularprice”. Chen,Marmorstein,TsirosandRao(2012)

Theoptimaldesignofacoupondependsuponthecustomer’spsychologicalorientation. RamanathanandDhar(2010)

Freesamplescanleadtopositivelong-termdemandeffects. BawaandShoemaker(2004)

Asequenceofincreasingcustomerbonusesismoreeffectiveatdrivingusageandretentionthanasequenceofdecreasingbonuses.

HaisleyandLoewenstein(2011)

Longerdeadlinesongiftcardsleadtolowerredemptionrates. ShuandGneezy(2010)

Uncertaintyaboutafreegiftcaninfluencetheeffectivenessofpromotion,dependinguponwhethercustomersareprimedtomakeemotionalorthoughtfuldecisions.

LaranandTsiros(2013)

Returningpromotedpricestoregularlevelsgraduallywillgeneratemoredemand. TsirosandHardesty(2010)

Promotionsthatextendstoretripscanleadtoanincreaseinunplannedexpenditure. Hui,Inman,HuangandSuher(2013)

Trainingcustomerstoappreciatethebenefitsoftheproductcanbemoreeffectivethanpromotions.

Gaurav,ColeandTobacman(2011)

PriceasaSignal

9-digitpriceendingscansignalapriceisdiscounted,particularlyfornewproducts. AndersonandSimester(2003)

“Sale”signsarealesscrediblesignalwhentheyareusedtoooften. AndersonandSimester(2001b)

Higherpricessethigherexpectations,andsocustomersevaluatealowqualityproductmorenegativelywhenithasahigherprice.

Gneezy,GneezyandLauga(2014)

Promotionscanlowerdemandbysignalingthataproductissuitableforcustomerswhoaremoresensitivetoqualitythanprice.

AndersonandSimester(2001a)

28|P a g e

Usingdiscountstoattractfirst-timecustomerscanmakethesecustomersmoreloyalbysignalingthatthefirmoffersgoodvalue.

AndersonandSimester(2004)

Multi-PartTariffsandOtherSchemes

Customersevaluateoutcomeswithinaperiod,ratherthanacrosstheentirecontract. LambrechtandTucker(2012)

Amulti-partpricingsystemcanlowerdemandifcustomersinterpretitasunfair. AndersonandSimester(2008)

Whendesigningarevenuemaximizing2-parttariffinthetelecommunicationsmarketitisimportanttoaccountforbothusageandcustomerretention,assolelyfocusingonusagewillsubstantiallyunder-estimatethesensitivityofrevenuetoprice.

Danaher(2002)

Consumersaresurprisinglyshort-termfocusedwhenchoosingbetweenlinearpriceschemesand3-parttariffs.

Yao,Mela,ChiangandChen(2012)

Howdocustomersrespondtoapay-what-you-wantpricingscheme? Kim,NatterandSpann (2009)

AdvertisingIssues

Isthereanimpactonpurchasing?

IttakesalotofdatatomeasuretheimpactofInternetadvertisingonconsumerspending. LewisandRiley(2014)

Retargetingisgenerallyineffective,unlessconsumersareclosetopurchasing. LambrechtandTucker(2013)

DVR'sdonotappeartoaffecthouseholdspending. Bronnenberg,DubeandMela(2010)

Competitors’advertisingcanincreaseyourdemand. AndersonandSimester(2013)

Whichmessagesaremosteffective?

Framinganappealasaself-predictionincreasesitseffectiveness. Spangenberg,Sprott,GrohmanandSmith(2003)

Matchingthetypeofappealwithafocusoneitherindividualversussocialbenefitscanimproveadvertisingeffectiveness.

WhiteandSimpson(2013)

Highlightingthebenefitsoftargetingislesseffectivethanemphasizingreciprocityforfreeaccess.

Schumann,vonWangenheimandGroene(2014)

Controloverpersonalizationcanmakepersonalizedadvertisementsmoreeffective. Tucker(2014)

Marketswithmorecompetitorscanappearmoreattractivetoentrants,astheremustbesufficientdemandtoattractsomanycompetitors.

TuckerandZhang(2010)

Advertisingpopularityinformationcanbenefitnichebrands,becausecustomersrecognizethatthesebrandsmustofferhighqualitytoovercometheirnarrowreach.

TuckerandZhang(2011)

Appealstoexplicitlycompareanofferwithcompetingoffersproducesmorecautiousbiddingbehaviorinauctions.

DholakiaandSimonson(2005)

Theeffectofadvertisingonword-of-mouthisstrongerforthosewhodonotalreadyassociatethebrandwiththemessage.

BergerandSchwartz(2011)

Assertivemessagingismoreeffectiveifcustomersbelieveinthecause,butlesseffective Kronrod,GrinsteinandWathieu

29|P a g e

iftheyarenotyetconvinced. (2012)

Optimizingpaidsearchadvertising

Clicksandrevenuesfororganicsearcharehigherinthepresenceofpaidsearch. YangandGhose(2010)

Clicksdecreasewithsearchpositionbutconversionmayincreaseswithsearchposition. Agarwal,HosanagarandSmith(2011)

MarketResearchandProductDevelopmentIssuesMarketResearch

Meremeasurementofpurchaseintentionsmayincreasethecorrelationbetweenstatedpurchaseintentionsandactualpurchaseprobabilities.

Chandon,MorwitzandReinartz(2005)

Participationincustomersatisfactionsurveysincreasessubsequentpurchaselikelihoods. Borle,Dholakia,SinghandWestbrook(2007)

Multimediacomputerinteractionscanrealisticallyportraycustomer’sinformationsources,andprovideforecaststhatarenotsignificantlydifferentfromtraditionalmethods.

Urban,Hauser,Qualls,Weinberg,BohlmannandChicos(1997)

Anoptimalcontactmodelforincreasingresponseratestoonlinemarketingresearchpanels.

Neslin,Novak,BakerandHoffman(2009)

ProductSizesandBundling

Ifacustomerneedsafixedquantity,theywillpurchasemoreunitswhentheproductisinaroundpackagecomparedtoasquarepackageofequalvolume.

Krider,RaghubirandKrishna(2001)

Bundlingsubstitutesorunrelateditemsinanauctionislessprofitablethansellingthemseparately,butbundlingismoreprofitableforcomplementaryproducts.

LeszczycandHaubl(2010)

Multipleunitpromotions(e.g.2for$1.50)aremoreeffectivethansingleunitpromotions(75-centseach).

Wansink,KentandHoch(1998)

ValidatingOptimizationModels

PricingModels Categorypricingofautomobileparts. Mantrala,Seetharaman,Kaul,

GopalakrishnaandStam(2006)Scholarshipsanduniversityadmissiondecisions Belloni,Lovett,Bouldingand

Staelin(2012)

AdvertisingModels

Optimizingsearchengineadvertisingbids. SkieraandNabout(2013)

Anadvertisingcontactmodelforcatalogmailing. Simester,SunandTsitsiklis(2006)

Estimatingtheincrementalimpactofmarketingactivitiesindifferentchannels. LiandKannan(2014)

30|P a g e

Banneradvertisingoptimization. Urban,Liberali,MacDonald,BordleyandHauser(2014)

MarketResearchandProductDevelopmentModels

Increasingresponseratestoonlinemarketingresearchpanels. Neslin,Novak,BakerandHoffman(2009)

Forecastingcustomerdemandforreallynewproducts. Urban,Hauser,Qualls,Weinberg,BohlmannandChicos(1997)

OtherTopics

SalesforceOptimization

Coordinatingsalescallstomatchforecastsofcustomers’purchasingdecisionscanincreaseprofits.

Kumar,VenkatseanandReinartz(2008)

Salesforcesalescontestshouldincludemultipleprizewinners,andrankorderingpricesincontestswithmultiplewinnersdoesnotincreasesalesorrevenues.

Lim,AhearneandHam(2009)

Word-of-MouthandReferrals

Word-of-mouthismoreeffectiveatincreasingsalesifitiscomesfromrelativelylessloyalcustomers,andfromacquaintancesinsteadoffriends.

GodesandMayzlin(2009)

Whichcustomersshouldafirmtargetwhendesigningcustomerreferralprograms? Kumar,PetersonandLeone(2010)

ParticipationinanOnlineCommunity

Participationinanonlinecommunityleadstolessspendingandfewerlistings. Algesheimer,Borle,DholakiaandSingh(2010)

Participationinanonlinecommunitymakescustomersmorewillingtoengageinriskybiddingbehavior,particularlywhentiestoothercommunitymembersarestrong.

Zhu,Dholakia,ChenandAlgesheimer(2012)

UserscontributetoTwitterbecausetheycareabouthowpeopleperceivethem,ratherthanbecausetheyderiveintrinsicutilityfromtheactivity.

ToubiaandStephen(2013)

EncouragingPositiveBehaviors

Competitions,pledgesandincentivescanallimprovechildren’sfoodchoices. Raju,RajagopalandGilbride(2010)

Earmarkingearningscanincreasesavingsrates. SomanandCheema(2011)

Other

Remodelingaretailspaceincreasessalesmorefornewcustomersthanestablishedcustomers.

DaggerandDanaher(2014)

Auctionoutcomescanbeinfluencedbyshippingcosts,secretreserveprices,andotherauctionfeatures.

HaruvyandLeszczyc(2011)