whatyou*need*to*know*about http*eventcollector* - splunk€¦ · disclaimer* 2...

18
Copyright © 2016 Splunk Inc. Shakeel Mohamed So=ware Engineer, Splunk What You Need To Know About HTTP Event Collector

Upload: others

Post on 23-Aug-2020

0 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: WhatYou*Need*To*Know*About HTTP*EventCollector* - Splunk€¦ · Disclaimer* 2 During*the*course*of*this*presentaon,*we*may*make*forward*looking*statements*regarding*future* events*or*the*expected*performance*of*the*company.*We

Copyright  ©  2016  Splunk  Inc.  

Shakeel  Mohamed  So=ware  Engineer,  Splunk  

What  You  Need  To  Know  About  HTTP  Event  Collector  

Page 2: WhatYou*Need*To*Know*About HTTP*EventCollector* - Splunk€¦ · Disclaimer* 2 During*the*course*of*this*presentaon,*we*may*make*forward*looking*statements*regarding*future* events*or*the*expected*performance*of*the*company.*We

Disclaimer  

2  

During  the  course  of  this  presentaNon,  we  may  make  forward  looking  statements  regarding  future  events  or  the  expected  performance  of  the  company.  We  cauNon  you  that  such  statements  reflect  our  current  expectaNons  and  esNmates  based  on  factors  currently  known  to  us  and  that  actual  events  or  results  could  differ  materially.  For  important  factors  that  may  cause  actual  results  to  differ  from  those  contained  in  our  forward-­‐looking  statements,  please  review  our  filings  with  the  SEC.  The  forward-­‐looking  statements  made  in  the  this  presentaNon  are  being  made  as  of  the  Nme  and  date  of  its  live  presentaNon.  If  reviewed  a=er  its  live  presentaNon,  this  presentaNon  may  not  contain  current  or  

accurate  informaNon.  We  do  not  assume  any  obligaNon  to  update  any  forward  looking  statements  we  may  make.  In  addiNon,  any  informaNon  about  our  roadmap  outlines  our  general  product  direcNon  and  is  

subject  to  change  at  any  Nme  without  noNce.  It  is  for  informaNonal  purposes  only  and  shall  not,  be  incorporated  into  any  contract  or  other  commitment.  Splunk  undertakes  no  obligaNon  either  to  develop  the  features  or  funcNonality  described  or  to  include  any  such  feature  or  funcNonality  in  a  future  release.  

Page 3: WhatYou*Need*To*Know*About HTTP*EventCollector* - Splunk€¦ · Disclaimer* 2 During*the*course*of*this*presentaon,*we*may*make*forward*looking*statements*regarding*future* events*or*the*expected*performance*of*the*company.*We

What  the  HEC?  A  new  token-­‐based  JSON  API  for  events    Send  events  directly  from  anywhere  (servers,  mobile  devices,  IOT)    Easy  to  configure  /  works  out  of  the  box.      Easy  to  secure    Highly  performant,  scalable  and  available    

3  

Page 4: WhatYou*Need*To*Know*About HTTP*EventCollector* - Splunk€¦ · Disclaimer* 2 During*the*course*of*this*presentaon,*we*may*make*forward*looking*statements*regarding*future* events*or*the*expected*performance*of*the*company.*We

In  6.5,  HEC  returns  with  even  more  powers!  

Page 5: WhatYou*Need*To*Know*About HTTP*EventCollector* - Splunk€¦ · Disclaimer* 2 During*the*course*of*this*presentaon,*we*may*make*forward*looking*statements*regarding*future* events*or*the*expected*performance*of*the*company.*We

5  

In  Splunk  6.3:    

HEC  only  accepts  data  in  our  JSON  event  format  

Page 6: WhatYou*Need*To*Know*About HTTP*EventCollector* - Splunk€¦ · Disclaimer* 2 During*the*course*of*this*presentaon,*we*may*make*forward*looking*statements*regarding*future* events*or*the*expected*performance*of*the*company.*We

HEC  RAW  

6  

You  can  now  send  data  in  arbitrary  formats  to  HEC!    Useful  for  integra/on  with  exis/ng  systems  that  can  send  data  over  HTTP              

Page 7: WhatYou*Need*To*Know*About HTTP*EventCollector* - Splunk€¦ · Disclaimer* 2 During*the*course*of*this*presentaon,*we*may*make*forward*looking*statements*regarding*future* events*or*the*expected*performance*of*the*company.*We

7  

In  Splunk  6.3:    

Clients  do  not  know  if  events  have  been  indexed  

Page 8: WhatYou*Need*To*Know*About HTTP*EventCollector* - Splunk€¦ · Disclaimer* 2 During*the*course*of*this*presentaon,*we*may*make*forward*looking*statements*regarding*future* events*or*the*expected*performance*of*the*company.*We

Indexer  ACK  

8  

You  can  now  receive  an  acknowledgment  when  events  are  indexed*    Useful  for  reducing  data  loss  during  outages    *Flexibility  comes  at  a  cost          

Page 9: WhatYou*Need*To*Know*About HTTP*EventCollector* - Splunk€¦ · Disclaimer* 2 During*the*course*of*this*presentaon,*we*may*make*forward*looking*statements*regarding*future* events*or*the*expected*performance*of*the*company.*We

/services/collector  

{"ackId":"1"}  

/services/collector/ack  

{"acks":  {"1":  false}}  

{"event":"something  happened"}  

Indexer  ACK  

{"event":"some…"}  

INDEXED  /services/collector/ack  

{"acks":{"1":  true}  

HEC  

Page 10: WhatYou*Need*To*Know*About HTTP*EventCollector* - Splunk€¦ · Disclaimer* 2 During*the*course*of*this*presentaon,*we*may*make*forward*looking*statements*regarding*future* events*or*the*expected*performance*of*the*company.*We

10  

In  Splunk  6.3:    

Only  regex  extracNons  are  supported  

Page 11: WhatYou*Need*To*Know*About HTTP*EventCollector* - Splunk€¦ · Disclaimer* 2 During*the*course*of*this*presentaon,*we*may*make*forward*looking*statements*regarding*future* events*or*the*expected*performance*of*the*company.*We

Index  field  enhancements  

11  

JSON  Field  extracNon  finally  works  for  HEC!    You  can  now  specify  addiNonal  index  fields  separate  from  the  "event"  playload!  (JSON  endpoint  only)    Useful  for  search  /me  perf  improvements  as  well  as  for  supplying  event  custom  metadata            

Page 12: WhatYou*Need*To*Know*About HTTP*EventCollector* - Splunk€¦ · Disclaimer* 2 During*the*course*of*this*presentaon,*we*may*make*forward*looking*statements*regarding*future* events*or*the*expected*performance*of*the*company.*We

12  

In  Splunk  6.3    

Clients  must  set  the  auth  header  to  the  Splunk  token  

Page 13: WhatYou*Need*To*Know*About HTTP*EventCollector* - Splunk€¦ · Disclaimer* 2 During*the*course*of*this*presentaon,*we*may*make*forward*looking*statements*regarding*future* events*or*the*expected*performance*of*the*company.*We

Basic  Auth  

13  

You  can  now  use  Basic  Auth  to  authenNcate  your  HEC  requests    Useful  for  systems  that  do  not  support  custom  auth  header  values  but  can  support  Basic  Auth  via  the  URI  (Github  web  hooks)            

Page 14: WhatYou*Need*To*Know*About HTTP*EventCollector* - Splunk€¦ · Disclaimer* 2 During*the*course*of*this*presentaon,*we*may*make*forward*looking*statements*regarding*future* events*or*the*expected*performance*of*the*company.*We

14  

In  Splunk  6.3    

You  need  to  master  Splunk  FU    to  monitor  HEC  instances  

Page 15: WhatYou*Need*To*Know*About HTTP*EventCollector* - Splunk€¦ · Disclaimer* 2 During*the*course*of*this*presentaon,*we*may*make*forward*looking*statements*regarding*future* events*or*the*expected*performance*of*the*company.*We

HEC  Console  

15  

You  can  now  monitor  HEC  using  the  new  DMC  dashboard    Useful  for  ensuring  your  HEC  instances  are  performing  as  they  should,  and  for  detec/ng  anomalies  /  failures            

Page 16: WhatYou*Need*To*Know*About HTTP*EventCollector* - Splunk€¦ · Disclaimer* 2 During*the*course*of*this*presentaon,*we*may*make*forward*looking*statements*regarding*future* events*or*the*expected*performance*of*the*company.*We

And  that's  not  it  

Dedicated  HEC  semngs    

•  SSL  •  CORS  •  IP  restricNons  

   

Page 17: WhatYou*Need*To*Know*About HTTP*EventCollector* - Splunk€¦ · Disclaimer* 2 During*the*course*of*this*presentaon,*we*may*make*forward*looking*statements*regarding*future* events*or*the*expected*performance*of*the*company.*We

Learn  more!  

17  

   

h4p://splk.it/new_HEC          

Page 18: WhatYou*Need*To*Know*About HTTP*EventCollector* - Splunk€¦ · Disclaimer* 2 During*the*course*of*this*presentaon,*we*may*make*forward*looking*statements*regarding*future* events*or*the*expected*performance*of*the*company.*We

THANK  YOU