Improve  the  Publisher  Yield  by  Recommending  Floor  Price  in  Second  Price  Auction  in  Non-­‐Guaranteed  Display  Advertising  Marketplace  

 Amit  Jain  

amitjain@yahoo-­‐inc.com    

December  11,  2013    1  Introduction        At   Yahoo,  we   have   a   non-­‐guaranteed   display-­‐advertising   platform   called   Right  Media  Exchange   (RMX).   RMX   supports   various   pricing   types   for   advertiser.   In   one   particular  pricing  type  called  dynamic  CPM  (dCPM),  it  supports  the  second  price  auction  where  the  first  winner  pays  based  on  second  winner's  price.  But  we  have  observed  a  significant  gap  between   the   first   price   and   second   price   of   the   winners,   and   that   results   in   under  monetization  or  lower  yield  for  RMX  publishers.      This  problem  becomes  even  worse  as  more  and  more  advertisers  (especially  RTB-­‐D)  are  planning   to   use   dCPM.   Most   of   the   RTB-­‐D   bidders   have   access   to   vast   amount   of  publisher,   user   and   marketplace   information.   They   use   sophisticated   models   to  purchase   inventory  at   the   lowest  cost.  This  kind  of  unfair  advantage  utilized  by  RTB-­‐D  advertisers  creates  an  imbalance  in  non-­‐guaranteed  display  marketplace.  This  vast  trove  of   information   utilization   by   advertisers   (RTB-­‐D   especially)   is   causing   lower   yield   for  publishers.      There  is  well  established  literature  on  various  aspects  of  auction  theory[1].  Elbittar  et  al.  [2]  has   focused  on  reserve  price   in   first  price  auction  using  simulated  data.  They  have  concluded   that   the   reserve   price   in   second   price   auction   didn’t   perform   as   well   as  compared  to  first  price  auction  for  a  particular  application  domain  of  telecom  frequency  spectrum  band   allocation   auction.  More   recently,   Schwarz   et   al.   [3]   have   done   large-­‐scale   field  experiment  on   setting  up   reserve  price   in   “sponsored”   search  marketplace.  They   have   empirically   shown   increase   in   publisher   (Yahoo)   yield   in   sponsored   search  marketplace.   In   search   marketplace,   user   intent   is   more   explicit   due   to   presence   of  search  query.  While   in  display  marketplace,  user   intent   is  difficult  to  estimate  and  this  presents  a  challenge  in  using  the  same  solution  as  suggested  by  Schwarz  et  al.  [3].    In   this  course  project,   I  explored  the  feasibility  of  machine-­‐learned  models   for  reserve  price  recommendations  on  per  ad  call  basis  in  non-­‐guaranteed  display  marketplace.  This  paper   is  organized  as   follows:  Section  2  provided  the  details  about   the  data  collection  approach  for  this  project.  Section  3  provides  the  supervised  learning  algorithm  used  so  far   to  experiment  with   reserve  price.  Conclusions  and   the  next   steps  are  presented   in  Section  4.    

2  Dataset      This   is   the   most   important   aspect   of   the   project.   Fortunately,   I   have   permission   to  access   RMX   Ad   serving   data   on   Yahoo’s   Hadoop   grid.   For   every   ad   call,   RMX   emit  significant   amount   of   information   about   the   auction.   All   this   information   resides   in  Yahoo's  Hadoop  grid.   I  used  pig  programming  to  write  map-­‐reduce   jobs  and  extracted  fields  relevant  to  this  project.  Since  publisher  wants  to  set  reserve  price  to  improve  the  yield,   and   since   publisher   controls   the   supply   side   attributes,   so   I   extracted   following  supply  side  attributes  on  which  a  publisher  can  potentially  set  reserve  price:  1)  Section  Id  (A  kind  of  unique  id  for  each  website),  2)  Size  ID  (Display  slot  size  on  the  publisher’s  webpage),  3)  User  agent  (type  of  browser),  4)  Gender  of  the  user,  5)  Age  of  the  user,  6)  Segment  ID  (Each  user  is  categorized  into  a  bunch  of  segments  like  sports,  finance,  auto  insurance  etc),  7)  Publisher  ID,  8)  Geographic  region  id  of  the  user    For  the  above  8-­‐tuple  key,  I  collected  first  bid  and  second  bid  amount  when  the  winning  advertising  pricing  type  is  ‘dCPM’.  Since  RMX  generates  billions  of  events  every  day  for  several   hundreds   ad   networks   that   are   participating   in   RMX,   so   I   focused   only   one  particular  network  that  is  Yahoo  Ad  Network  (YAN).  Initially,  I  collected  data  in  the  batch  of   10   minutes,   and   every   such   batch   had   2-­‐3   millions   of   records.   Based   on   various  analyses  done  on   this   data,   I   figured  out   that   the  10  minutes   interval   is   not   the   right  batch  size  as  the  bid  landscape  changes  dramatically  over  the  period  of  10  minutes.    I  collected  4  training  sets:  one  batch  of  2  second  interval  (3869  records),  one  batch  of  5  second  interval  (16404  records),  one  batch  of  30  second  interval  (102108  records),  and  one  batch  of  1  minute  interval  (194K  records).  For  test  dataset,  I  collected  a  different  2-­‐minute  interval  with  377K  records.  All  this  data  is  captured  in  tab-­‐separated  file.    3  Machine  Learning  Algorithms    3.1  Naïve  Implementation    I   have  modeled   the   reserve   price   problem  as   the   supervised   learning   algorithm  using  linear  regression.  To  perform  the  linear  regression,  I  have  considered  all  the  supply  side  attributes  except  Segment  Id  as  mentioned  in  section  2.  The  approach  is  as  follows:       1.  Estimate  the  parameters  for  First  price  data  using  hypothesis  function  (linear  function)  in  7  parameters  (except  segment  ID)  as  follows:    

𝑯𝟏𝜽 𝒙 =  𝜽𝟏𝟎𝒙𝟎 +  𝜽𝟏𝟏𝒙𝟏 +⋯+  𝜽𝟏𝟕𝒙𝟕                                          -­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐(1)       2.  Estimate  the  parameters  for  Second  price  data  using  the  similar  hypothesis  as  in  equation  (1).    

  𝑯𝟐𝜽 𝒙 =  𝜽𝟐𝟎𝒙𝟎 +  𝜽𝟐𝟏𝒙𝟏 +⋯+  𝜽𝟐𝟕𝒙𝟕                  -­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐(2)       3.  Estimate  the  reserve  price  as  weighted  combination  of  first  price  and  second  price.       𝑹𝑷 𝒙 =  𝑯𝟏𝜽(𝒙)𝑾𝟎 +  𝑯𝟐𝜽(𝒙)𝑾𝟏                                                  -­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐  (3)          I  haven’t  estimated  the  weights  W0  and  W1.   I  have  used  W0=0  and  W1=1.1   i.e.  reserve  price   is   10%   more   than   the   estimated   second   bid   price.   Since   the   dataset   has   huge  number  of  records,  so  I  used  LMS  algorithm  using  stochastic  gradient  descent  approach  as  mentioned  in  Handout  #1  page  #7  of  CS229.      As  mentioned  in  section  2,  I  collected  2  batches  of  data  of  10  min  interval.  Each  of  the  batches  has  2  million   records  approximately.   I  used   the   first  batch  as   training   set  and  second   batch   as   test   set.   On   the   test   set,   I   simulated   the   revenue   gain   or   loss   by  applying  reserve  price  obtained  in  equation  (3).    

𝑹𝒆𝒗𝒆𝒏𝒖𝒆  𝑮𝒂𝒊𝒏 = 𝑹𝑷(𝒙𝒊)− 𝑺𝑷𝒊

𝒎

𝒊!𝟏

 

 Where  SPi  is  second  bid  price  in  ith  test  record.  m  is  number  of  test  data.    On  the  collected  test  data,  revenue  gain  was  positive  but  only  $5  that  means  using  the  reserve  price,  publishers  will  make  $5  more  in  every  10  minute  which  is  very  minuscule  %  increase   in  yield.  These  results  were  not  satisfactory  as  the  %  increase   in  publishers  yield  is  expected  to  be  5%-­‐30%.      When  I  trained  this  model  on  the  4  training  sets  mentioned  in  Section  2,  and  then  test  on   the   test   dataset,   it   exhibited   the   high   variance   behavior   i.e.  worked   quite  well   on  training  dataset,  but  worked  very  poorly  on  test  dataset.      3.2  Second  Model  with  Only  One  Supply  Side  Attribute    To  overcome  the  high  variance   issue  of  model  developed   in  section  3.1,   I  used  one  of  the  most  salient  attributes  called  “size”  to  develop  the  linear  regression  model.    Estimate  first  price:        𝑯𝟏𝜽 𝒙 =  𝜽𝟏𝟎𝒙𝟎 +  𝜽𝟏𝟏𝒙𝟏  Estimate  second  price:    𝑯𝟐𝜽 𝒙 =  𝜽𝟐𝟎𝒙𝟎 +  𝜽𝟐𝟏𝒙𝟏    I  also  changed  the  error  measure  from  revenue  gain  to  sum  of  square  error.  First  price  error:     = 𝑯𝟏𝜽(𝒙𝒊)− 𝑭𝑷𝒊

𝒎𝒊!𝟏

2  

Second  price  error:     = 𝑯𝟐𝜽(𝒙𝒊)− 𝑺𝑷𝒊𝒎𝒊!𝟏

2  

 Where  𝑭𝑷𝒊  is  first  bid  price,  SPi  is  second  bid  price  in  ith  test  record.  m  is  number  of  test  data.  

 I  plotted   training  and   test  error  using   the  4   training  dataset,  and   it  exhibited   the  high  bias  behavior  as  expected.    Training  error  increased  with  increasing  sample  size,  and  test  error   decreased   gradually,   and   both   kinds   of   error   were   more   than   the   desired  performance  level.    3.3  Third  Model  with  Three  Supply  Side  Attributes  (Publisher  Id,  Section  Id,  Size  Id)    I  decided  to  use  three  salient  supply  side  attributes  namely  Publisher  Id,  Section  Id,  and  Size  id  to  find  a  middle  ground  between  high  variance  and  high  bias  scenario  of  model  1  (section   3.1)   and   model   2   (section   3.2)   respectively.   But   the   results   came   out   to   be  exactly   similar  as  model  2   (section  3.2).   So   it  meant  with  addition  of  Publisher   Id  and  Section  Id  to  the  model,  the  mutual  information  gain  was  zero.        4  Conclusions    I   found  out   that  addition  of  all   the  7  attributes   in   the  model   resulted   in  high  variance  scenario.  Similarly,  having  a  simplistic  model  with   just  one  parameter  namely  “size   id”  resulted  in  high  bias  cases.  Although  data  was  collected  for  “segment  ID”  as  one  of  the  supply  side  attribute,  but  it  was  never  considered  in  any  of  the  three  models  discussed  in   section  3  due   to  very  high  cardinality  and   inconsistency   in   the  data.    And  based  on  recent   published   articles   on   several   advertising   blogs,   I   came   to   know   that   RTB-­‐D  bidders  buy  the  inventory  mainly  for  premium  audience  segments.  So  segment  ID  could  have  played  an  important  role  in  improving  linear  regression  based  models.  As  a  future  work,  one  of  the  recommendations  will  be  to  improve  the  data  quality  for  segment  IDs,  and   then   include   them   as   one   of   the   modeling   parameter.     I   could   not   able   to   get  approval   to   test   these  models   in   live  production   traffic.  So  as  a  next  step,   I’ll  work  on  creating  a  feedback  loop  by  launching  the  experiment  in  1%  bucket,  and  then  collecting  data  and  pushing  out  new  models  to  ad  serving  system.        References    [1]  http://en.wikipedia.org/wiki/Auction_theory  [2]  Elbittar,  A.,  Unver,  M.  (2003),  On  Determination  of  Optimal  Reserve  Price  in  Auctions  with  Common  Knowledge  about  Ranking  of  Valuations.  [3]  Michael  Ostrovsky,  Michael  Schwarz  (2009),  Reserve  Prices  in  Internet  Advertising  Auctions:  A  Field  Experiment.  [4]  Reserve  price  in  Google’s  DoublieClick  Ad  Server.  http://doubleclickpublishers.blogspot.com/2011/11/introducing-­‐minimum-­‐cpm-­‐recommendations.html