INFRASTRUCTURE ARCHITECTUREGUSTAVO GARNICA

INFRASTRUCTURE ARCHITECTURE

AGENDA

▸ Introduction

▸ Semantics

▸ Architecture

▸ Engineering

▸ Conclusions

INFRASTRUCTURE ARCHITECTURE

INTRODUCTION

INFRASTRUCTURE ARCHITECTURE

ABOUT

▸ Walk the walk in software development, architecture and infrastructure

▸ Also into ethics and moral philosophy

▸ Self-taught dude

▸ Been there, done that

▸ Fully committed to automation and self-service IT

INFRASTRUCTURE ARCHITECTURE

HAVE YOU DRIVEN ON I-10?

▸ Cross-country US interstate highway

▸ From SR-1 in Santa Monica, CA to I-95 in Jacksonville, FL

▸ 3,959.53 KM across CA, AZ, NM, TX, LA, MS, AL, FL

▸ Part of US infrastructure plans of 1959

INFRASTRUCTURE ARCHITECTURE

CHECK IT OUT…

INFRASTRUCTURE ARCHITECTURE

CHECK IT OUT…

INFRASTRUCTURE ARCHITECTURE

CHECK IT OUT…

INFRASTRUCTURE ARCHITECTURE

ROADS ARE INFRASTRUCTURE, RIGHT?

▸ What do you see …

Nothing?, great!, that is the whole point

INFRASTRUCTURE ARCHITECTURE

NOW CONSIDER THIS …

INFRASTRUCTURE ARCHITECTURE

OR NEARER, THIS …

INFRASTRUCTURE ARCHITECTURE

WHAT ARE WE TALKING ABOUT?

▸ Role of infrastructure in development

▸ Extremely expensive in the physical world

▸ Readily available in information technology

▸ Start-ups can compete

▸ Engineers can easily train themselves

INFRASTRUCTURE ARCHITECTURE

CAN YOU DIG IT?

Can you count … ?

I say, the future is ours... if you can count!

INFRASTRUCTURE ARCHITECTURE

WE MEAN …

▸ Amazon S3 transactions per second: 1.1 million

▸ USD 30.4 billion

▸ Google searches daily: 3.3 billion

▸ USD 21.5 billion

▸ Facebook daily active users: 1.13 billion

▸ USD 6.43 billion

▸ Netflix daily served video-hours: 125 million

▸ USD 1.96 billion

INFRASTRUCTURE ARCHITECTURE

SEMANTICS

INFRASTRUCTURE ARCHITECTURE

SEMANTICS

▸ Sort of a natural sequence …

RELIABILITY GROWTH SCALABILITY

COMPLEXITY REDUNDANCY

AVAILABILITY

TESTING RISK RISK

INFRASTRUCTURE ARCHITECTURE

CHECKPOINT

▸ Success depends largely on availability

▸ Availability comes ahead of reliability

▸ Availability is manageable

▸ Scalability serves availability

INFRASTRUCTURE ARCHITECTURE

ARCHITECTURE

INFRASTRUCTURE ARCHITECTURE

WHAT CAN HURT YOUR INFRASTRUCTURE?

▸ Miscalculation

▸ Dependencies

▸ Technical debt

▸ Poor choices

▸ Growth

INFRASTRUCTURE ARCHITECTURE

WHAT CAN YOU DO?

▸ Decompose

▸ Anticipate failure

▸ Consider risk

▸ Quantify

▸ Architect accordingly

INFRASTRUCTURE ARCHITECTURE

DECOMPOSE

▸ Consider your dependencies

▸ What does an end-to-end transaction look like?

▸ What are good candidates for decoupling?

▸ Identify problematic components

▸ Where can we implement redundancy?

▸ Be conscious of your limitations

▸ Understand your resources

▸ Know your SLAs

INFRASTRUCTURE ARCHITECTURE

ANTICIPATE FAILURE

▸ Failure will occur

▸ What are the logical and physical limits?

▸ Consider what if and when scenarios

▸ Emphasis on critical areas

INFRASTRUCTURE ARCHITECTURE

CONSIDER RISK

▸ Document your points of failure

▸ Consider both severity and likelihood

▸ Assess the risks

▸ Cost of removing a risk

▸ Cost of recovering from a materialized risk

▸ Plan mitigation

INFRASTRUCTURE ARCHITECTURE

QUANTIFY

▸ Measure everything

▸ Calculate target uptime

▸ Consider planned downtime and historical unscheduled downtime

▸ Gauge your work

▸ How much computing power do you need?

▸ What is the cost of your infrastructure?

▸ Establish your baseline

▸ How do business projections impact your baseline?

INFRASTRUCTURE ARCHITECTURE

ARCHITECT

▸ Require logical partitioning to enable physical partitioning

▸ Each problematic or high risk component must be redundant

▸ Calculate redundancy carefully *

▸ Support transactional isolation

▸ Implement a caching tier, including a CDN

▸ All multi-user content must be cached

INFRASTRUCTURE ARCHITECTURE

ARCHITECT

▸ Leverage reduced or null state in the middle tiers

▸ Equalize computing type to application orientation

▸ Always let clients talk to workers

▸ Shoot for limited scale, beyond 10x re-architect

▸ Divide and conquer your traffic load

▸ Provision at least one environment that is equivalent to production

INFRASTRUCTURE ARCHITECTURE

ENGINEERING

INFRASTRUCTURE ARCHITECTURE

MOVE FORWARD

▸ You have your architecture, now …

▸ Leverage the cloud

▸ Automate everything

▸ Implement end-to-end monitoring

▸ Roll your own

INFRASTRUCTURE ARCHITECTURE

LEVERAGE THE CLOUD

▸ Your cloud

▸ Easily implementable resilience

▸ Extreme scalability

▸ Virtualisation versus containerization

▸ Your SLA, not your cloud’s

▸ The physical world still applies

INFRASTRUCTURE ARCHITECTURE

AUTOMATE EVERYTHING

▸ All changes, all the time

▸ Provisioning and deployments

▸ Include recovery from failure!

▸ Reduce human factor

▸ Build certainty and repeatability

▸ Reject manual work on production systems

▸ Big effect on availability

INFRASTRUCTURE ARCHITECTURE

END-TO-END MONITORING

▸ Focused monitoring

▸ Circuit-break on dependencies

▸ Thorough

▸ Runtime

▸ Configuration

▸ Alerting

▸ Identify trends

INFRASTRUCTURE ARCHITECTURE

GREAT PROJECTS

▸ Risk mitigation

▸ Thunderstorm (Facebook)

▸ Chaos Monkey (Netflix)

▸ Roll your own

▸ HHVM (Facebook)

▸ MapReduce (Google)

▸ Network gear (Amazon)

INFRASTRUCTURE ARCHITECTURE

CONCLUSIONS

▸ Infrastructure can be a competitive advantage

▸ In IT, hope should not be a strategy

▸ Embrace the reality of failure

▸ Automate yourself out of the job

▸ Build or tweak your platform to your unique needs

▸ Be open to evolve (and throw-away) your infrastructure

INFRASTRUCTURE ARCHITECTURE

QUESTIONS

www.linkedin.com/in/gustavogarnica

@lonetechie

INFRASTRUCTURE ARCHITECTURE

REFERENCES▸ Definition of morality from Stanford Encyclopaedia of Philosophy (http://plato.stanford.edu/entries/morality-definition/)

▸ Interstate 10 in Wikipedia (https://en.wikipedia.org/wiki/Interstate_10)

▸ Favela Petare in Caracas Venezuela (http://www.dronestagr.am/favelas-of-petare-neighborhood-caracas-venezuela-6/)

▸ Cyrus, The Warriors, 1979 (http://www.imdb.com/title/tt0080120/quotes)

▸ Google Knowledge Graph (http://www.theverge.com/2016/10/4/13122406/google-phone-event-stats)

▸ Facebook results 2016 Q2 (https://investor.fb.com/investor-news/press-release-details/2016/Facebook-Reports-Second-Quarter-2016-Results/default.aspx)

▸ Facebook 2016 Q2 revenue (https://s21.q4cdn.com/399680738/files/doc_financials/2016/Facebook-Reports-Second-Quarter-2016-Results.pdf)

▸ Netflix company profile (https://ir.netflix.com/)

▸ Netflix 2016 Q2 revenue (https://ir.netflix.com/results.cfm)

▸ Amazon Blog - 2013! (https://aws.amazon.com/blogs/aws/amazon-s3-two-trillion-objects-11-million-requests-second/)

▸ Amazon 2016 Q2 revenue (http://www.businessinsider.com/amazon-earnings-q2-2016-2016-7)

▸ High Availability in Wikipedia (https://en.wikipedia.org/wiki/High_availability)