scaling drupal on amazon web services dr
TRANSCRIPT
Scaling Drupal on
Amazon Web Services
Tristan Roddis DrupalCamp Brighton, January 2015
What we created: www.qdl.qa
2
3
4
5
• Several hundred thousand pages
• Tens of terabytes of data
• Requirement for high throughput
• Requirement for high resilience
6
What is Drupal?
7
Let’s scale it!
8
Let’s not scale it like this!
9
Centralised database
10
Centralised filesystem
11
Avoid single point of failure
12
AWS TLAs
• EC2 – Elastic Compute Cloud = virtual servers
• RDS – Relational Database Service = database
• ELB – Elastic Load Balancer = distribute requests
13
Therefore…
14
RDS has easy redundancy
15
So now we have
16
Clustered filesystem: GlusterFS
17
GlusterFS components
• EBS – Elastic Block Storage
= file storage up to 1Tb
• LVM – Logical Volume Manager
= grow and shrink volumes
• Gluster has storage ‘bricks’
• Cluster appears as single mount point
(NFS, Gluster protocol etc.)
18
Distribute servers
• Amazon has different Regions and
Availability Zones
19
Use different Availability Zones
20
I lied…
21
Adding Solr and IIPImage…
22
Additional security
• VPC – Virtual Private Cloud = can create your
own network architecture
• Security Groups = firewall rules
• IAM = Identity and Access Management =
restrict AWS roles
23
Final setup
24
Woah – that’s complicated
25
Server provisioning
• Automate everything!
• Use Ansible (or Chef, Puppet, etc.)
26
“If you have to SSH into your servers,
then your automation has failed.”
— Rich Adams
27
Pets vs cattle
28
“When one of them gets sick, you shoot
'em in the head and replace 'em with a
new one.”
—Randy Bias, CEO of Cloudscaling
Deployment I: Drupal tools
• Drush
• Drush master
• Features
• Strongarm
29
Deployment II
• Use Fabric (or Ansible, Capistrano, etc.)
• @runs_once decorator for e.g. drush commands.
30
Load Test
• Jmeter
• Blitz.io or similar
31
Tune performance
• Tools: Apache Benchmark (ab), Yslow, Google
PageSpeed
• xhprof – remove bottlenecks
• Fix warnings and notices
• Disable database logging
• OpCode cache (Zend Optimizer, APC)
• Minify CSS/JS
• Lazy Loading via AJAX
32
33
Add caching
• Varnish
• Redis/memcached for cache tables (can use
Amazon Elasticache)
• CDN (e.g. Amazon Cloudfront)
• Re-run load tests
34
[auto]scaling
• AMIs = Amazon Machine Image
• Autoscaling policies
• “pets versus cattle” again
• Problem: servers must have latest Drupal code
• Conflict with Fabric 'push‘ technique
• Interim solution: create an AMI after
each deployment
35
Backup
• AMIs - Amazon Machine Image = snapshot
server images
• ELB snapshots
• S3 – Simple Storage Service = REST storage
• Glacier = long-term storage
• We used:
– Snapshots with RDS
– S3 with versioning, plus Glacier for Drupal
assets and image assets
36
Monitoring I: Cloudwatch alerts
37
Monitoring II: log analysis
• SumoLogic to aggregate logs (or Loggly,
Papertrail, etc.)
38
Monitoring III: notifications
• OpsGenie (or PagerDuty or similar)
39
Gotchas
• GlusterFS for PHP files – don’t!
• Custom Drupal cache – careful of
uncontrolled growth!
40
Tips
• Use “termination protection”
• Enable ELB logging
• “Maintenance 200” module
• Create a “down for maintenance” server
41
Conclusion
• 21 instances
• 6 load balancers
• 67 ELB volumes
• Lot of work to set up, but quick to alter
• Easily coped with launch traffic:
– 21k sessions/day
– peak of 350 concurrent users
42
Further Reading
• Justin Slattery, "Multiple Region Autoscaling Drupal in Amazon Web Services" http://dev.mlsdigital.net/posts/Cloud-Native-Drupal/
• Rich Adams, "AWS Tips I Wish I'd Known Before I Started" https://wblinks.com/notes/aws-tips-i-wish-id-known-before-i-started/
• Laura Hamilton, "Are your servers pets or cattle?" http://www.lauradhamilton.com/servers-pets-versus-cattle
43
www.cogapp.com/careers
