aws summit berlin 2013 - "angrybirds fly in the cloud" - scaling and market needs for the...
Post on 20-Aug-2015
1.304 Views
Preview:
TRANSCRIPT
Challenges
Frequent development and releasing of new
features
Scalable and stable Operation
Freitag, 3. Mai 13
8 regions with 2-4 availability zonesA lot of „edge servers“ for content delivery
Freitag, 3. Mai 13
What you should do
Automate the setup of new servers
Automate failover
Automate scaling
Freitag, 3. Mai 13
What you should do
Automate the setup of new servers
Automate failover
Automate scaling
Don‘t do work that others did better
Freitag, 3. Mai 13
Powerfull AWS Services for Angrybirds
Amazon Web Services & Amazon Console
CloudFront ElastiCacheS3EC2 RDS ELB R53
Freitag, 3. Mai 13
Powerfull AWS Services for Angrybirds
Rightscale Services
Amazon Web Services & Amazon Console
CloudFront ElastiCacheS3EC2 RDS ELB R53
Freitag, 3. Mai 13
RDS
Varnish Array
Frontend Array Backend Array
RDS
S3Assets Storage
EC2 EC2EC2 EC2 EC2
EC2 EC2
Freitag, 3. Mai 13
RDS
Varnish Array
Frontend Array Backend Array
RDS
CloudFrontCDN for Assets
S3Assets Storage
EC2 EC2EC2 EC2 EC2
EC2 EC2
Freitag, 3. Mai 13
RDS
Varnish Array
Frontend Array Backend Array Worker Array
RDS
CloudFrontCDN for Assets
S3Assets Storage
EC2 EC2 EC2EC2 EC2 EC2
EC2 EC2
Freitag, 3. Mai 13
RDS
Varnish Array
Frontend Array Backend Array Worker Array
ELB
RDS
CloudFrontCDN for Assets
S3Assets Storage
EC2 EC2 EC2EC2 EC2 EC2
EC2 EC2
Freitag, 3. Mai 13
RDS
„Deployment Definition“ + Autoscaling
Varnish Array
Frontend Array Backend Array Worker Array
ELB
RDS
CloudFrontCDN for Assets
S3Assets Storage
EC2 EC2 EC2EC2 EC2 EC2
EC2 EC2
Freitag, 3. Mai 13
RDS
„Deployment Definition“ + Autoscaling
Varnish Array
Frontend Array Backend Array Worker Array
ELB
RDS
CloudFrontCDN for Assets
S3Assets Storage
EC2 EC2 EC2EC2 EC2 EC2
EC2 EC2Monitoring
Freitag, 3. Mai 13
RDS
„Deployment Definition“ + Autoscaling
Varnish Array
Frontend Array Backend Array Worker Array
ELB
RDS
CloudFrontCDN for Assets
S3Assets Storage
EC2 EC2 EC2EC2 EC2 EC2
EC2 EC2MonitoringAPI
Freitag, 3. Mai 13
RDS
„Deployment Definition“ + Autoscaling
Varnish Array
Frontend Array Backend Array Worker Array
CloudFrontCDN for Skin
ELB
RDS
CloudFrontCDN for Assets
S3Assets Storage
EC2 EC2 EC2EC2 EC2 EC2
EC2 EC2MonitoringAPI
Freitag, 3. Mai 13
RDS
„Deployment Definition“ + Autoscaling
Varnish Array
Frontend Array Backend Array Worker Array
CloudFrontCDN for Skin
ELBR53DNS
RDS
CloudFrontCDN for Assets
S3Assets Storage
EC2 EC2 EC2EC2 EC2 EC2
EC2 EC2MonitoringAPI
Freitag, 3. Mai 13
RDS
„Deployment Definition“ + Autoscaling
Varnish Array
Frontend Array Backend Array Worker Array
CloudFrontCDN for Skin
ELBR53DNS
RDSElastiCache
Cache Backend
CloudFrontCDN for Assets
S3Assets Storage
EC2 EC2 EC2EC2 EC2 EC2
EC2 EC2MonitoringAPI
Freitag, 3. Mai 13
RDS
„Deployment Definition“ + Autoscaling
Varnish Array
Frontend Array Backend Array Worker Array
CloudFrontCDN for Skin
ELBR53DNS
RDSS3
DeploymentElastiCache
Cache Backend
CloudFrontCDN for Assets
S3Assets Storage
EC2 EC2 EC2EC2 EC2 EC2
EC2 EC2MonitoringAPI
Freitag, 3. Mai 13
CPU
Network
Storage (DB)Traffic
Possible BottlenecksOffloading, Optimisation, Caching, add more CPU (scale)...
Offloading, Optimisation, Scale Storage UP (H/V), Replace Storage...
Offloading (CDN, Proxy)Optimize Size of the SiteScale Network IS...
Freitag, 3. Mai 13
HTTP Caching
Appl
icat
ion
Offload expensive requests
Traffic
CDN
Brow
ser
Traffic
Freitag, 3. Mai 13
HTTP Caching
Appl
icat
ion
Varnish (Reverse Proxy) Traffic
Offload expensive requests
Traffic
CDN
Brow
ser
Traffic
Freitag, 3. Mai 13
HTTP Caching
Appl
icat
ion
Varnish (Reverse Proxy) Traffic
Offload expensive requests
Traffic
CDN
Brow
ser
TrafficPurging
Freitag, 3. Mai 13
Autoscaling
•Groups with policys•Min, Max•Votingrules for scaling up & down•Replace broken instances
Traffic
Fron
tend
Arr
ay
EC2
EC2
EC2
EC2
EC2
EC2
Freitag, 3. Mai 13
And you?
• Do you use continuous integration?• Do you use automated acceptance tests?
Freitag, 3. Mai 13
And you?
• Do you use continuous integration?• Do you use automated acceptance tests?• Do you keep them green?
Freitag, 3. Mai 13
And you?
• Do you use continuous integration?• Do you use automated acceptance tests?• Do you keep them green?• Do you use continuous deployment?
Freitag, 3. Mai 13
Continuous Deployment
We want on-demand deploymentof tested features and fixesin a quick and reliable way
Freitag, 3. Mai 13
Commitand Build
Build Downstream and Test Strategy
SVN Backup Storage
Angry Birds Deployment Pipeline
Feedback to Developers
Freitag, 3. Mai 13
Commitand Build
Build Downstream and Test Strategy
SVN Backup Storage
Unit Tests
Angry Birds Deployment Pipeline
Feedback to Developers
Freitag, 3. Mai 13
Commitand Build
Build Downstream and Test Strategy
SVN Backup Storage
Unit Tests
Angry Birds Deployment Pipeline
Feedback to Developers
Freitag, 3. Mai 13
Commitand Build
Build Downstream and Test Strategy
SVN Backup Storage
Unit Tests Install on “latest”
Angry Birds Deployment Pipeline
Feedback to Developers
Freitag, 3. Mai 13
Commitand Build
Build Downstream and Test Strategy
SVN Backup Storage
Unit Tests Install on “latest”
Angry Birds Deployment Pipeline
Feedback to Developers
Freitag, 3. Mai 13
Selenium Acceptance Tests
Commitand Build
Build Downstream and Test Strategy
SVN Backup Storage
Unit Tests Install on “latest”
Angry Birds Deployment Pipeline
Feedback to Developers
Freitag, 3. Mai 13
Selenium Acceptance Tests
Commitand Build
Build Downstream and Test Strategy
SVN Backup Storage
Unit Tests Install on “latest”
Angry Birds Deployment Pipeline
Feedback to Developers
Freitag, 3. Mai 13
Selenium Acceptance Tests
Commitand Build
Build Downstream and Test Strategy
SVN Backup Storage
Unit Tests Install on “latest”
Performance Tests
Angry Birds Deployment Pipeline
Feedback to Developers
Freitag, 3. Mai 13
Selenium Acceptance Tests
Commitand Build
Build Downstream and Test Strategy
SVN Backup Storage
Unit Tests Install on “latest”
Performance Tests
Angry Birds Deployment Pipeline
Feedback to Developers
Freitag, 3. Mai 13
Angry Birds Deployment Pipeline
Install on Deployment
SystemCopy to S3
Cloud Deployment
Staging
Integration Tests
AOE + Rovio
Cloud deployment production
Freitag, 3. Mai 13
Varnish Array
AutoscalingFrontend Array
Backend Array
Worker Array
Cloud Deployment
ELB
Deployment „Version 88“
Start new release by✓ cloning old deployment✓ adjusting release number
RDSmySQL S3
V V V
WFE FE FE BE BE BE
R53DNS
Freitag, 3. Mai 13
Varnish Array
AutoscalingFrontend Array
Backend Array
Worker Array
Cloud Deployment
ELB
Deployment „Version 88“
RDSmySQL S3
V V V
WFE FE FE BE BE BE
R53DNS
Freitag, 3. Mai 13
Deployment „Version 89“
Varnish Array
AutoscalingFrontend Array
Backend Array
Worker Array
Cloud Deployment
ELB
Deployment „Version 88“
RDSmySQL S3
Varnish Array
AutoscalingFrontend Array
Backend Array
Worker Array
V V V
WFE FE FE BE BE BE
R53DNS
Freitag, 3. Mai 13
Deployment „Version 89“
Varnish Array
AutoscalingFrontend Array
Backend Array
Worker Array
Cloud Deployment
ELB
Deployment „Version 88“
RDSmySQL S3
Varnish Array
AutoscalingFrontend Array
Backend Array
Worker Array
ELB
W
V V V
WFE FE FE BE BE BE
R53DNS
Freitag, 3. Mai 13
Deployment „Version 89“
Varnish Array
AutoscalingFrontend Array
Backend Array
Worker Array
Cloud Deployment
ELB
Deployment „Version 88“
RDSmySQL S3
Varnish Array
AutoscalingFrontend Array
Backend Array
Worker Array
ELB
WFE FE FE BE BE BE
V V VV V V
WFE FE FE BE BE BE
R53DNS
Freitag, 3. Mai 13
Deployment „Version 89“
Varnish Array
AutoscalingFrontend Array
Backend Array
Worker Array
Cloud Deployment
ELB
Deployment „Version 88“
RDSmySQL S3
Varnish Array
AutoscalingFrontend Array
Backend Array
Worker Array
ELB
WFE FE FE BE BE BE
V V VV V V
WFE FE FE BE BE BE
R53DNS
Freitag, 3. Mai 13
New release deployed
Space game launchedArray scales up
Day 3 - „People want it so badly“
Freitag, 3. Mai 13
New release deployed
Space game launchedArray scales up
USA wakes up
Day 3 - „People want it so badly“
Freitag, 3. Mai 13
40% average CPU usage.
1-2 FE Servers are enough for normal operations
Day 4 - Release 3 „Performance Improvement“
Freitag, 3. Mai 13
Thanks!Follow us on Twitter:
AOE: @aoemediaDaniel P: @danpoetz
Flickr photo contributions in the slide:http://www.flickr.com/photos/the-wanderers-eye/4494147652/sizes/l/http://www.flickr.com/photos/51293088@N05/8195327886/sizes/l/http://www.flickr.com/photos/k1rsch/3818553037/sizes/l/
Freitag, 3. Mai 13
top related