building scalable websites for the cloud
DESCRIPTION
Brian Adler, Solutions Consultant at RightScale, led this session at the RightScale User Conference 2010 in Santa Clara. Session Abstract: RightScale has extensive experience building scalable websites from the ground up. More importantly, we have invaluable experience rescuing companies who have come to us after their self-built websites failed to scale when hit with unexpected traffic levels. In this session, we'll review reference architectures and share best practices from proven, scalable websites deployed in the cloud. Additionally, we'll cover alternate methods for load balancing along with techniques to improve your web application's availability and reliability.TRANSCRIPT
![Page 1: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/1.jpg)
1
Scalable Web ApplicationsReference Architectures and Best Practices
Brian Adler, PS Architect
![Page 2: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/2.jpg)
2
Scalable Web Application
![Page 3: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/3.jpg)
3
Scalable Web Application• What?
• An application built on an architecture that can adapt to changing conditions
![Page 4: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/4.jpg)
4
Scalable Web Application• What?
• An application layered on an architecture that can adapt to changing conditions
• Why?• Traffic and load patterns are unpredictable
• Viral or flash-mob events can result in very dynamic conditions
• Availability and Reliability• Application must be distributed to increase likelihood of end-user accessibility
• Overprovision• Under-utilized resources == wasted $$$
• Underprovision• Missed opportunities – users unable to access your site/product• Don’t be a victim of your own success
![Page 5: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/5.jpg)
5
This bed is too big. This bed is too small…
![Page 6: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/6.jpg)
6
Cloud Resource Model• Dynamically provision the resources you need to meet the
current demand• Demand goes up, resources are added• Demand goes down, resources are removed
• In true “utility computing” fashion, only pay for what you use, when you use it
![Page 7: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/7.jpg)
7
But this bed is just right
![Page 8: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/8.jpg)
8
Scalable Web Application• When?
• No time like the present
![Page 9: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/9.jpg)
9
Scalable Web Application• When?
• No time like the present
• How?• Stay tuned…
![Page 10: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/10.jpg)
10
Reference Architecture
![Page 11: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/11.jpg)
11
Load Balancing Tier
![Page 12: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/12.jpg)
12
Load Balancing• ELB or not ELB. That is the question.
• No SSL termination on the ELB (*)• Can load balance at the TCP level, but that eliminates sticky sessions for
secure connections (*)• (*) No longer the case as of mid-October 2010
• Can scale to handle large amounts of traffic, but slow to ramp-up• Only need one• (RightScale has a technical white paper on load balancing solutions that
can be provided if desired)
![Page 13: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/13.jpg)
13
Load Balancing• ELB or not ELB. That is the question.
• No SSL termination on the ELB (*)• Can load balance at the TCP level, but that eliminates sticky sessions for
secure connections (*)• (*) No longer the case as of mid-October 2010
• Can scale to handle large amounts of traffic, but slow to ramp-up• Only need one• (RightScale has a technical white paper on load balancing solutions that
can be provided if desired)
• HAProxy + Apache• Can handle SSL termination on the load balancer• Allows for sticky sessions with secure connections• Each instance can handle a specific amount of traffic with no ramp-up time• Each instance can only handle a specific amount of traffic• Addition of load balancers is possible, but requires DNS modifications
![Page 14: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/14.jpg)
14
Load Balancing• Load Balancer + Application server
• Possible, and good for test and dev• Not a best practice for a production environment
• Traffic spikes can cause instance to perform both load balancing and application functions…poorly
![Page 15: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/15.jpg)
15
Load Balancing• Load Balancer + Application server
• Possible, and good for test and dev• Not a best practice for a production environment
• Traffic spikes can cause instance to perform both load balancing and application functions…poorly
• Recommendation: Minimum of two load balancers• Each load balancer should be in a different availability zone (AZ) to
increase reliability and availability• RightScale testing has shown that m1.large is a good choice for load
balancers• Due to 100K-120K packet-per-second limit, larger instances do not provide much gain in
throughput• Roughly 5K responses/second can be handled by m1.large• With the 5K threshold in mind, select the number of load balancers required to handle your
peak traffic
![Page 16: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/16.jpg)
16
Application Server Tier• Puts the “scalable” in a scalable application• True autoscaling a must in any dynamic/unpredictable
environment
![Page 17: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/17.jpg)
17
Application Server Tier• Autoscaling
• Fully automated server launch based on autoscaling triggers• No manual intervention (can be challenging in certain environments, i.e.
Windows)• Download and install application code from common repository to ensure
identical configuration of all servers in the tier
![Page 18: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/18.jpg)
18
Application Server Tier• Autoscaling
• Fully automated server launch based on autoscaling triggers• No manual intervention (can be challenging in certain environments, i.e.
Windows)• Download and install application code from common repository to ensure
identical configuration of all servers in the tier
• Triggers• Common
• CPU idle• Free memory• System load
• Custom• Web server connections• Application-specific metrics
![Page 19: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/19.jpg)
19
Application Server Tier• When to scale?
• Conservatively. Both up and down
![Page 20: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/20.jpg)
20
Application Server Tier• When to scale?
• Conservatively. Both up and down
• Up• Allow adequate lead time for new servers to become operational• Before system is negatively impacted• Look for trends in activity and react early• Worst that can happen: Charged for an extra instance hour
![Page 21: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/21.jpg)
21
Application Server Tier• When to scale?
• Conservatively. Both up and down
• Up• Allow adequate lead time for new servers to become operational• Before system is negatively impacted• Look for trends in activity and react early• Worst that can happen: Charged for an extra instance hour
• Down• When system has been underutilized for a consistent, consecutive period
of time• Scale down fewer servers than in a scale-up event• Again, only downside is an extra hour of instance charge• Better safe than sorry
![Page 22: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/22.jpg)
22
Application Server Tier• Array considerations
![Page 23: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/23.jpg)
23
Application Server Tier• Array considerations
• Weight the array across all availability zones (not regions)• Increases reliability of application• NOTE: Traffic within an AZ on private IPs is free. Traffic between AZs incurs a per-gigabyte
charge• Traffic between regions is charged at public Internet rates
![Page 24: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/24.jpg)
24
Application Server Tier• Array considerations
• Weight the array across all availability zones (not regions)• Increases reliability of application• NOTE: Traffic within an AZ on private IPs is free. Traffic between AZs incurs a per-gigabyte
charge• Traffic between regions is charged at public Internet rates
• Set minimums and maximums appropriately• Minimum can assist in cost savings in times of low usage• Maximum can limit overall cost exposure
![Page 25: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/25.jpg)
25
Application Server Tier• Array considerations
• Weight the array across all availability zones (not regions)• Increases reliability of application• NOTE: Traffic within an AZ on private IPs is free. Traffic between AZs incurs a per-gigabyte
charge• Traffic between regions is charged at public Internet rates
• Set minimums and maximums appropriately• Minimum can assist in cost savings in times of low usage• Maximum can limit overall cost exposure
• Instance size• m1.large is typically a good choice for array-based servers in a production environment
• m1.smalls (and even micro instances) can be used in test and development environments
• Every application is different, so run load tests and benchmarks to find the optimal solution for your environment
![Page 26: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/26.jpg)
26
Application Server Tier• Array considerations
• Weight the array across all availability zones (not regions)• Increases reliability of application• NOTE: Traffic within an AZ on private IPs is free. Traffic between AZs incurs a per-gigabyte
charge• Traffic between regions is charged at public Internet rates
• Set minimums and maximums appropriately• Minimum can assist in cost savings in times of low usage• Maximum can limit overall cost exposure
• Instance size• m1.large is typically a good choice for array-based servers in a production environment
• m1.smalls (and even micro instances) can be used in test and development environments
• Every application is different, so run load tests and benchmarks to find the optimal solution for your environment
• Code Deployment• Updated code can be pushed to all servers in an array via a single click of a button
![Page 27: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/27.jpg)
27
Caching Tier• Caching can dramatically decrease the load on the database
• Particularly in read-intensive applications
• Costs of caching• Application complexity/modification• Additional instance hours to support the cache
![Page 28: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/28.jpg)
28
Caching Tier• Best practice is to have a separate, dedicated caching tier
• Caching can be implemented on each application server• Prevents the use of a distributed cache• Local cache should only be used by the co-resident application server
• Application complexities• Database performance degradation
![Page 29: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/29.jpg)
29
Caching Tier• Best practice is to have a separate, dedicated caching tier
• Caching can be implemented on each application server• Prevents the use of a distributed cache• Local cache should only be used by the co-resident application server
• Application complexities• Database performance degradation
• Instance Size and Count• Determine memory caching footprint
• Select instance size and count that spreads the load over several servers• Prevents loss of entire cache if a single instance fails• Distribute caching servers across AZs for reliability• Overprovision if possible
• Provide capacity for system to grow to fully utilize cache (budget permitting)
![Page 30: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/30.jpg)
30
Caching Tier• Best practice is to have a separate, dedicated caching tier
• Caching can be implemented on each application server• Prevents the use of a distributed cache• Local cache should only be used by the co-resident application server
• Application complexities• Database performance degradation
• Instance Size and Count• Determine memory caching footprint
• Select instance size and count that spreads the load over several servers• Prevents loss of entire cache if a single instance fails• Distribute caching servers across AZs for reliability• Overprovision if possible
• Provide capacity for system to grow to fully utilize cache (budget permitting)
• Manually scaling caching servers is possible but non-trivial• Involves application and database performance degradation• Time To Lives (TTLs)
• Always set to expire
![Page 31: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/31.jpg)
31
Caching Tier• Write-intensive applications
• Don’t see as large a performance gain as read-intensive apps• Memcached can be modified to perform lazy writes of data objects to the
database• Data can be lost in case of caching server crash• Not a recommended best practice, but can be (and has been) done• Tradeoff of performance versus end-user experience
![Page 32: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/32.jpg)
32
Caching Tier• Write-intensive applications
• Don’t see as large a performance gain as read-intensive apps• Memcached can be modified to perform lazy writes of data objects to the
database• Data can be lost in case of caching server crash• Not a recommended best practice, but can be (and has been) done• Tradeoff of performance versus end-user experience
• Third-party providers• Vendor solutions exist that allow dynamic memcached scaling
![Page 33: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/33.jpg)
33
Database Tier• Numerous database architecture options exist• No “one size fits all” solution, so testing and benchmarking are
critical to determine proper configuration
![Page 34: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/34.jpg)
34
Database Tier• Masters and Slave(s)
• Multiple Slaves if budget permits• Distribute Master and Slave(s) across AZs• Always use same instance size for Master and Slaves
![Page 35: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/35.jpg)
35
Database Tier• Masters and Slave(s)
• Multiple Slaves if budget permits• Distribute Master and Slave(s) across AZs• Always use same instance size for Master and Slaves
• Data Storage• EBS volumes for data store• Never use ephemeral storage for persistent data• Back up Master and Slaves frequently• Upload snapshots to S3 or some other persistent, redundant storage
![Page 36: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/36.jpg)
36
Database Tier• Masters and Slave(s)
• Multiple Slaves if budget permits• Distribute Master and Slave(s) across AZs• Always use same instance size for Master and Slaves
• Data Storage• EBS volumes for data store• Never use ephemeral storage for persistent data• Back up Master and Slaves frequently• Upload snapshots to S3 or some other persistent, redundant storage
• Instance Size• Varies greatly based on the nature of the application and site traffic• Load testing and benchmarking can assist in identifying a reasonable
initial size
![Page 37: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/37.jpg)
37
Database Scaling
![Page 38: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/38.jpg)
38
Database Scaling• Vertical
• “Grow” or “shrink” a database server from one instance size to another
![Page 39: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/39.jpg)
39
Database Scaling• Vertical
• “Grow” or “shrink” a database server from one instance size to another
• Horizontal• Add additional servers to spread the database load
![Page 40: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/40.jpg)
40
Database Vertical/Horizontal Scaling
RightScale Makes Vertical or Horizontal Scaling Easier
SmallType A
SmallType B
SmallType B
SmallType B
SmallType A
SmallType A
LargeType A
LargeType B
HorizontalScaling
VerticalScaling
More servers of same types
Same quantity of larger servers
![Page 41: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/41.jpg)
41
Horizontal Database Scaling• Addition of read Slaves
• Effective for read-intensive applications• Only writes need to access the master• Replication lag to slaves must be considered
![Page 42: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/42.jpg)
42
Horizontal Database Scaling• Addition of read Slaves
• Effective for read-intensive applications• Only writes need to access the master• Replication lag to slaves must be considered
• Effective mechanism is to use MySQL Proxy
![Page 43: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/43.jpg)
43
Horizontal Database Scaling• Sharding
• Concept is to partition the database into distinct, non-overlapping pieces• “Horizontal slicing” of the database tables into groups of rows• Forethought required in setting up shards since cross-shard joins are
resource intensive
![Page 44: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/44.jpg)
44
Horizontal Database Scaling
Before Sharding
![Page 45: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/45.jpg)
45
Horizontal Database Scaling
After Sharding
![Page 46: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/46.jpg)
46
Horizontal Database Scaling• Master-Master
• Two (or more) Master DBs• Any Master can modify any database object• Replication lag can result in database inconsistencies• Poorly-designed applications can cause data object collisions and leave
databases in indeterminate state• Not a recommended best practice, nor supported by RightScale
![Page 47: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/47.jpg)
47
Horizontal Database Scaling• NoSQL solutions
• Many options exist – SimpleDB, Cassandra, Membase, CouchDB, MongoDB, Riak, etc.
• Basically a Key/Value store• No complex operations between data objects (no relational operations)• Multiple nodes can be used to implement a distributed data store
• Coordinated backup and recovery can be challenging
• Some RightScale customers are beginning to use some NoSQL solutions in specific use cases.
![Page 48: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/48.jpg)
48
So What’s Best?
![Page 49: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/49.jpg)
49
So What’s Best?• As is typical in many technology discussions…
![Page 50: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/50.jpg)
50
So What’s Best?• As is typical in many technology discussions…
“It depends”
![Page 51: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/51.jpg)
51
So What’s Best?• As is typical in many technology discussions…
• Many scalable environments share some common underlying architecture concepts
“It depends”
![Page 52: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/52.jpg)
52
So What’s Best?• As is typical in many technology discussions…
• Many scalable environments share some common underlying architecture concepts
• Every application is different. As such, there is no “one size fits all”
“It depends”
![Page 53: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/53.jpg)
53
So What’s Best?• As is typical in many technology discussions…
• Many scalable environments share some common underlying architecture concepts
• Every application is different. As such, there is no “one size fits all”
• Components of a reference architecture such as this can be used as a starting point, with tweaks and modifications made per the unique characteristics of the application itself, or the load and traffic patterns it experiences
“It depends”
![Page 54: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/54.jpg)
54
Scalable Web Applications
Q&ARightScale.com/Conference
(Presentations available next week)
RightScale.com/whitepapers
![Page 55: Building Scalable Websites for the Cloud](https://reader035.vdocument.in/reader035/viewer/2022062709/558e01ca1a28ab8d6c8b45a1/html5/thumbnails/55.jpg)
55