scalable, good, cheap · get your infrastructure started right! (not just preparing for incident...
TRANSCRIPT
![Page 1: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/1.jpg)
Scalable, Good, Cheap
a tale of sexiness, puppets, shell scripts, and python
![Page 2: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/2.jpg)
From this...
![Page 3: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/3.jpg)
![Page 4: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/4.jpg)
...to this!
![Page 5: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/5.jpg)
![Page 6: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/6.jpg)
Get your infrastructure started right!
(not just preparing for incident and rapid event response)
![Page 7: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/7.jpg)
Who we are?
Avleen Vig (@avleen)Senior Systems Engineer at EtsyGood at: Scaling frontends, pythonPrevious companies: WooMe, Google, Earthlink
Marc Cluet (@lynxman)Senior Systems Engineer at WooMeGood at: Backend scaling, bash/python, languagesPrevious companies: RTFX, Tiscali, World Online
![Page 8: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/8.jpg)
Overview
Workflow
Why planning for scaling is important
How do you choose your software
Setting up your infrastructure
Managing your infrastructure
![Page 9: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/9.jpg)
The background
Larger startup, $32m in funding6 million+ active usersDozens of developers6 systems administrators4 DBAs10+ code releases every dayGeographically distributed employees
Brooklyn HQSatellites in Berlin, San FranciscoSmall number of remote employees
![Page 10: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/10.jpg)
The background
Small, funded start up6 python developers2 front end developers3 systems administrators1 DBA (moustache included)Multiple code releases every dayGeographically distributed employees
Berlin, Copenhagen, Leeds, London, Los Angeles, Oakland, Paris, Portland, Zagreb
![Page 11: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/11.jpg)
Workflow
Ticket systemsTicket, or it didn't happen!
DocumentationWikis are good
Don't Repeat YourselfIf you keep doing the same thing manually, automate
Version control everythingAll of your scriptsAll of your configurations
![Page 12: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/12.jpg)
Workflow
Everything will change
Technical debt vs Premature optimisationIf you try to be too accurate too early, you'll fail
![Page 13: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/13.jpg)
Team integration
Be sure to hire the right peopleBeer recruitment interview
Encourage speedRelease soon and release often
Embrace mistakes as part of your day to dayLearn to work with it
Ask for peer reviews for important componentsHelps sanity checking your logic
Developers, Sysadmins, DBAs, one team
![Page 14: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/14.jpg)
Team communication
Team communication is the most critical factor
Make sure everyone is in the loop
Useful applicationsIRCSkypeemailshout!
Don't be afraid to use the phone to avoid miscommunication
![Page 15: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/15.jpg)
Layering! Not just for haircuts.
Separate your systems
Front end
Application
Database
Caching
![Page 16: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/16.jpg)
Choosing your software
What does your software need to do?FastCGI / HTTP proxy? Use nginxPHP processing? Use apache
What expertise do you already have?Stick to what you're 100% good at
Don't rewrite everythingIf it does 70% of what you need it's good for you
![Page 17: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/17.jpg)
Release management
Fast and furious
Automate, automate, automate
Script your deploys and rollbacks
Continuous deployment
MTTR vs MTBF
![Page 18: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/18.jpg)
MTTR vs MTBF
![Page 19: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/19.jpg)
![Page 20: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/20.jpg)
Logging
Centralize your logging
syslog-ng
Parsing web logs - the secret troubleshooting weapon
SQL
Splunk
![Page 21: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/21.jpg)
Web logs in a database!CREATE TABLE access ( ip inet, hostname text, username text, date timestamp without time zone, method text, path text, protocol text, status integer, size integer, referrer text, useragent text, clienttime double precision, backendtime double precision, backendip inet, backendport integer, backendstatus integer, ssl_cipher text, ssl_protocol text, scheme text);
![Page 22: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/22.jpg)
Web logs in a database!
![Page 23: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/23.jpg)
Monitoring
Alerting vs Trend analysis
![Page 24: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/24.jpg)
![Page 25: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/25.jpg)
Monitoring
Alerting vs Trend analysisNagios is great for raising alerts on problems
![Page 26: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/26.jpg)
![Page 27: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/27.jpg)
Monitoring
Alerting vs Trend analysis
Nagios is great for raising alerts on problems
Ganglia is great at long term trend analysis
Know when something is out of the "ordinary"
![Page 28: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/28.jpg)
Monitoring
Alerting vs Trend analysis
Nagios is great for raising alerts on problems
Ganglia is great at long term trend analysis
Know when something is out of the "ordinary"
What should you monitor?
Anything which breaks once
Customer facing services
![Page 29: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/29.jpg)
Monitoring
Alerting vs Trend analysis
Nagios is great for raising alerts on problems
Ganglia is great at long term trend analysis
Know when something is out of the "ordinary"
What should you graph?
Everything! If it moves, graph it.
Customer facing rates and statistics
![Page 30: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/30.jpg)
Monitoring
Get statistics from your logs:
PostgreSQL: pgfouine
MySQL: mk-query-digest
Web servers: webalizer, awstats, urchin
Custom applications: Do it yourself! Integrate with Ganglia
![Page 31: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/31.jpg)
Monitoring
![Page 32: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/32.jpg)
Caching
Caches are disposable
![Page 33: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/33.jpg)
![Page 34: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/34.jpg)
Caching
Caches are disposable
But what about the thundering herd?
![Page 35: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/35.jpg)
The importance of scaling
![Page 36: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/36.jpg)
The importance of scaling
August 2003 Northeastern US and Canada blackout
Caused by poor process execution
Lack of good monitoring
Poor scaling
![Page 37: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/37.jpg)
The importance of scaling
![Page 38: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/38.jpg)
The importance of scaling
Massive destruction avoided!
256 power stations automatically shut down
85% after disconnecting from the grid
Power lost but plants saved!
![Page 39: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/39.jpg)
Caching
Caches are disposable
But what about the thundering herd?
Increase backend capacity along with cache capacity
Plan for cache failure
Reduce demand when cache fails
![Page 40: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/40.jpg)
Caching
Find out how your caching software works
Memcache + peep!
Is it better with lots of keys and small objects?
Or fewer keys and large objects?
How is memory allocated?
![Page 41: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/41.jpg)
Caching
Caches are disposableSolved!
But what about the thundering herd?Solved!
Now we get into database scaling!Over to Marc...
![Page 42: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/42.jpg)
Databases
Databases...
or how to live and die dangerously
![Page 43: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/43.jpg)
Databases
SQL or NoSQL?
![Page 44: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/44.jpg)
Databases
SQLGives you transactional consistencyGood known systemHard to scale
NoSQLTransactionally consistent "eventually"New cool systemEasy to scale
![Page 45: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/45.jpg)
Databases
SQLGives you transactional consistencyGood known systemHard to scale
NoSQLTransactionally consistent "eventually"New cool systemEasy to scale
You may end up using BOTH!
![Page 46: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/46.jpg)
Databases
Be smart about your table design
![Page 47: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/47.jpg)
Databases
Be smart about your table designKeep it simple but modular to avoid surprises
![Page 48: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/48.jpg)
You need to design your database right!
![Page 49: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/49.jpg)
Databases
Be smart about your table designKeep it simple but modular to avoid surprisesDon't abuse many-to-many tables, they will just give you hell
![Page 50: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/50.jpg)
Databases
Be smart about your table designKeep it simple but modular to avoid surprisesDon't abuse many-to-many tables, they will just give you hell
YOU WILL GET IT WRONGYou'll need to redesign parts of your DB semi-regularlyBe prepared for the unexpected
![Page 51: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/51.jpg)
Databases
The read dilemma
As the tables grow so do read times and memory.Several options:
Check your slow query log, tune indexes
Partition to read smaller numbers of rows
Master / Slave, but this adds replication lag!
![Page 52: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/52.jpg)
Databases
The read dilemma
As the tables grow so do read times and memory.Several options:
Check your slow query log, tune indexesSingle most common problem with slow queries and capacityBe careful about foreign keys
![Page 53: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/53.jpg)
Databases
The read dilemma
As the tables grow so do read times and memory.Several options:
Check your slow query log, tune indexes
Partition to read smaller numbers of rowsBy range (date, id)By hash (usernames)By anything you can imagine!
![Page 54: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/54.jpg)
Databases
The write conundrum
As the database grows so do writes
Writes are bound by disk I/ORAID1+0 helps
Don't shoot yourself in the foot!Don't try to solve this earlyHave monitoring ready to foresee this issueBring pizza
![Page 55: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/55.jpg)
Databases
Divide writes! Remember about modular? This is it
![Page 56: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/56.jpg)
Databases
How to give a consistent view to the servers?
Use a query director!
pgbouncer on Postgres
gizzard on MySQL
![Page 57: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/57.jpg)
Web frontend
Hardware load balancers - Good but expensive!
Software load balancers - Good and cheap! (more pizza)
Web server frontendsnginx, lighttpd, apache
Reverse proxiesvarnish, squid
Kernel stuffLinux ipvs
![Page 58: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/58.jpg)
Web frontend
Which way should I go?
Web servers as load balancersGives you nice add on featuresYou can offload some process in the frontendBuffering problems
Reverse proxiesCaching stuff is goodFast reaction timeNo buffering problems
![Page 59: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/59.jpg)
Web frontend
Divide your web clusters!
You can send different requests to different clusters
You can use an API call to connect between them
![Page 60: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/60.jpg)
Configuration management
Be ready to mass scaleKeep all your machines in line
Automated server installsUse it to install new softwareAlso to rapidly deploy new versions
![Page 61: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/61.jpg)
Writing tools
If you do something more than 2 times it's worth scripting
Write small tools when you need them
Stick to one or two languagesAnd be good at them
![Page 62: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/62.jpg)
Writing tools
Even better
Have your scripts repo in a cvs and push it everywhere
![Page 63: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/63.jpg)
Backups
It's important to have backups
![Page 64: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/64.jpg)
Backups
It's important to have backups
It's even more important to exercise them!Having backups without testing recovery is like having no backups
![Page 65: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/65.jpg)
Backups
It's important to have backups
It's even more important to exercise them!Having backups without testing recovery is like having no backups
How can we exercise backups for cheap?
![Page 66: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/66.jpg)
Backups
It's important to have backups
It's even more important to exercise them!Having backups without testing recovery is like having no backups
How can we exercise backups for cheap?Cloud computing!
![Page 67: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/67.jpg)
Cloud computing
Cloud computing help us recreate our platform on the cloud
Giving us a more than credible recovery scenario
Also very useful to spawn more instances if we run into problems
![Page 68: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/68.jpg)
Interesting things to read
Wikipediahttp://en.wikipedia.org/wiki/DevOps
Web Operations and Capacity Planninghttp://kitchensoap.com
High scalability (if you get there)http://highscalability.com/
If you really fancy databases, explain extendedhttp://explainextended.com/
![Page 69: Scalable, Good, Cheap · Get your infrastructure started right! (not just preparing for incident and ... Why planning for scaling is important How do you choose your software Setting](https://reader033.vdocument.in/reader033/viewer/2022050101/5f40c2fd40d7bd42c92b643c/html5/thumbnails/69.jpg)
Questions?
Work at Etsy!http://etsy.com/jobs
Work at WooMe!http://bit.ly/work4woome
@lynxman@avleen