cloud concepts
DESCRIPTION
This is a presentation i have done in WSO2. I have referred and used slides from Azeez's slides at http://www.slideshare.net/afkham_azeez/java-colombo-developing-highly-scalable-apps?qid=65a88726-9fbd-44d6-ae16-36eecffdcc9f&v=default&b=&from_search=2TRANSCRIPT
Last Updated: May. 2014
Committer and PMC member of Apache StratosSenior Software Engineer, WSO2
Lahiru Sandaruwan
Cloud Concepts
*
Agenda
• Some core concepts• Vertical/horizontal scaling techniques• Capacity planning• SLA Awareness• Autoscaling• Fault tolerance• Q&A, Discussion
*
Scalability
"The ability of the of a system to continue to operate correctly even when it is scaled to a larger size”
*
Availability
*
Availability
*
High Availability
A system that is designed for continuous operation in the event of a failure of one or more components. However, the system may display some degradation of service, but will continue to perform correctly.
High Availability: The proportion of time during which the service is accessible with reasonable response times should be close to 100%.
*
How to decide required scale (capacity) & availability?
• Average throughput (TPS)• Max throughput (TPS)• Monetary value of a transaction• Average loss & max loss per second of
downtime• Decide on how much to invest based on cost vs.
benefit tradeoff
*
Vertical Scaling
• Get the maximum out of each allocated JVM or resource
• Increase CPU size• Increase memory
*
Horizontal Scaling
*
Load Balancing
• Load balancing algorithms• Round robin• Weighted• Response based• Health check• Failover-only
*
Clustering for scalability
*
Clustering for availability
Group Communication Channel/State replication
*
Capacity Planning
Source: http://srinathsview.blogspot.com/2012/05/how-to-measure-performance-of-server.html
*
Capacity Planning
Throughput = number of completed requests / time to complete the requests
No. of servers = (projected max load * 1.3) / max throughput of one server
max throughput of one server = max throughput of that server for the slowest scenario in the set of use cases
*
Static membership
• Predefined members• Other (non-predefined) nodes cannot join
M2M1
M3 M4
Static group
N
*
Dynamic membership
• No predefined members• Nodes can join & leave
M2M1
M3 M4
Dynamic group
NJoin
*
Hybrid membership• Some predefined (well-known) members, and some
dynamic members• Nodes can join & leave• Membership revolves around the static members
Hybrid group
N
M2M1
M3 M4
Static members
M6M5
M7
Dynamic members
Join (IP, Port)
*
Multicast based membership management
N
M2
M1
M3
M4
Join (IP, Port)
*
Well-known Address (WKA) based membership management
Hybrid group
NWK2
WK1
WK3 WK4
Static members
M6M5
M7
Dynamic members
Join (IP, Port)
Notify
*
Multicast vs. WKA
Multicast WKA
All nodes should be in the same subnet Nodes can be in different networks
All nodes should be in the same multicast domainNo multicasting requirement
Multicasting should not be blocked
No fixed IP addresses or hosts required At least one well-known IP address or host required
Failure of any member does not affect membership discovery
New members can join with some WKA nodes down, but not if all WKA nodes are down
Does not work on IaaSs such as Amazon EC2 IaaS-friendly
Requires keepalived, elastic IPs or some other mechanism for remapping IP addresses of WK members in cases of failure
*
● Solves performance, availability, and economic costs issues
● Currently the customer needs a considerable capacity planning
● Reduce cost of cloud: reduce economic cost and energy footprint
SLA Awareness
*
● Monitoring the cloud parameters● Trigger an event if a SLA violation happened● Acting upon the events
SLA Awareness: Solution
*
Auto-scaling
• Scale-out when load increases• Scale-in when load decreases• Always use the optimum amount of resources• Try out
• AWS ELB • Apache Stratos Load Balancer• WSO2 ELB
*
Auto-scaling – steady load
*
Auto-scaling – load increasing
*
Auto-scaling – load increasing
*
Auto-scaling – steady load
*
Auto-scaling – decreased load
*
Auto-scaling – decreased load
*
Auto-scaling – steady load
*
Autoscaling - Analysis & Results
*
Autoscaling - Analysis & Results
*
Single node
Availability
Cost
LOWEST
HIGHEST
*
Primary-secondary
Primary Secondary
Availability
Cost
LOWEST
HIGHEST
*
Primary-secondary, multiple LB
Primary Secondary
keepalived
Availability
Cost
LOWEST
HIGHEST
*
Active cluster, multiple LB
Active Active
keepalived
Active
Availability
Cost
LOWEST
HIGHEST
*
Multi-zone or multi-datacenter Deployment
Region X
Zone 1
Zone 2
Cloud Controller
Availability
Cost
LOWEST
HIGHEST
*
Multi-region deployment
Region X
Zone 1
Zone 2
Region Y
Zone 1
Zone 2
Availability
Cost
LOWEST
HIGHEST
*
Multiple IaaS (hybrid) DeploymentAvailability
Cost
LOWEST
HIGHEST
Private cloud (data center)
Zone 1
Zone 2
Amazon EC2
Zone 1
Zone 2
Rackspace Cloud
Zone 1
Zone 2
*
Sin
gle
Nod
e
Prim
ary-
Sec
onda
ry, s
ingl
e LB
Mul
ti-no
de a
ctiv
e cl
uste
r- S
ingl
e zo
ne
Mul
ti-re
gion
Prim
ary-
Sec
onda
ry,
with
mul
tiple
LB
s
Mul
ti-zo
ne Mul
ti-Ia
aS
Cost of Availability
*
Stratos Architecture
*
References
๏ http://www.cari-info.org/cari-2012/session%201/1B3.pdf๏ http://domino.research.ibm.com/library/cyberdig.
nsf/papers/16D9BBF221540A0785257784004FC33D๏ http://ieeexplore.ieee.org/xpl/login.jsp?
tp=&arnumber=5452504&url=http%3A%2F%2Fieeexplore.ieee.org%2Fxpls%2Fabs_all.jsp%3Farnumber%3D5452504
๏ http://ieeexplore.ieee.org/xpl/login.jsp?tp=&arnumber=5990687&url=http%3A%2F%2Fieeexplore.ieee.org%2Fxpls%2Fabs_all.jsp%3Farnumber%3D5990687
๏ http://www.slideshare.net/afkham_azeez/java-colombo-developing-highly-scalable-apps
- Azeez’s talk at Java Colombo
*
DISCUSSION
*
Thanks!