failing well: managing risk in high performance applications

Presented  by  Alison  Giano1o  Foocamp  2013  

MANAGING RISK IN HIGH-PERFORMANCE APPS

FAILING …

ABOUT ME •  I  am  the  CTO/CSO  for  noise.  

•  We  build  stuff  for  brands  like  Int

el,  

vitaminwater,  JPMorgan  Chase,  GE,  Sunkist,  

Sears,  Unilever,  &  more.  

•  Managing  risk  is  a  big  part  of  wha

t  I  do.  

•  Risk  management  doesn’t  have  to  be  boring.  

Hi, I’m Alison!

FOOCAMP 2013

@snipeyhead  @snipeyhead  

Risk is not BAD or GOOD.

It just IS, and it will exist

whether you choose to

acknowledge it or not.

FOOCAMP 2013

THIRD-PARTY DEPENDENCIES

FOOCAMP 2013

RISK:

(APIs,  SaaS/PaaS)  

APPLICATION SECURITY

FOOCAMP 2013

RISK:

(XSS,  CSRF,  SQL  Injec<on,  et

c.)  

APPLICATION PERFORMANCE

FOOCAMP 2013

RISK:

(Code  Errors,  Technical  Deb

t,  Bad  

Queries,  Cache  Failures)  

SERVER PERFORMANCE

FOOCAMP 2013

RISK:

(Misconfigura<ons,  Exceeding

Capacity,  Hardware/Netwo

rk  

Failure)  

BRITTLE DEPLOYMENT

FOOCAMP 2013

RISK:

(Problems  during  deployment,  

par<ally  deployed  code)  

OVERLY COMPLEX SYSTEMS

FOOCAMP 2013

RISK:

(Impossible  to  debug,  difficult  to  

change)  

RISK-MATRIX •  Every  project,  

every  Sme.  No  excuses.  

•  Create  a  risk  matrix    

•  Probability  of  failure  

•  Impact  of  failure  

•  Ways  to  minimize  probability  

•  Game  plan  if  failure  occurs  

FOOCAMP 2013

TRANSPARENCY •  All  stakeholder

s  sign-­‐off  on  the  risk  

matrix  

•  Manages  expectaSons    

•  Gives  you  a  clear  course  of  acS

on  during  

crisis  

FOOCAMP 2013

AKA “COVERING YOUR ASS”

Just because something has

risk doesn’t necessarily mean

it’s a bad decision.

FOOCAMP 2013

Sometimes, the risk is worth

the reward.

FOOCAMP 2013

FOOCAMP 2013

Different  perspecSves  can  uncover  more  risk    Counters  the  sSgma  of  risk  being  BAD    Prevents  the  sSgma  of  being  the  one  who  always  says  NO!  

Risk Management is

a TEAM EFFORT!

LOG EVERYTHING

FOOCAMP 2013

TIP:

(Automate  log  parsing  to  alert  if  

there’s  trouble)  

MONITOR ALL THE THINGS!

FOOCAMP 2013

TIP:

(Design  your  system  with  

monitoring  in  mind)  

PREMATURE OPTIMIZATION

FOOCAMP 2013

TIP:

(JUST  SAY  NO!)  

GET TO KNOW YOUR USERS

FOOCAMP 2013

TIP:

(If  something  changes,  there’s  

probably  a  reason)  

FEWER MOVING PARTS IS

BETTER

FOOCAMP 2013

TIP:

THANK YOU!

FOOCAMP 2013

@snipeyhead