microsoft: building a massively scalable system with datastax and microsoft's next generation...
TRANSCRIPT
Building a massively scalable system with DataStax and
Microsoft's next generation Paas infrastructure
Rob Bagby Sr. Cloud Architect – Microsoft
www.deveducate.com / www.robbagby.com
@RBagby
Jesus AguilarSr. Cloud Architect – Microsoft
www.giventocode.com
@GivenToCode
How do I scale my Data linearly?Does the VM have the required resources?
Are there any cold VMs that I shouldn't be paying for?
How do I move a service?
Is the VM running?Is the VM too hot?How do I deploy across a part of the cluster?
Does the VM have the required capacity?
How do I roll back a deployment if it is does not work?
How do I deploy across a part of the cluster?
Can I have multiple versions running side-by-side?
Where should I move it?
Does my data have a single point of failure?Is my application running?
Can I replicate data across data centers?
Large and Distributed Challenges
Ops Management High Density
Resource Governance
Deployment and Packaging
Rolling updates
Scalability
Hyper Scale
Load balancing
Scale Out
Maintainability
Granular versioning
In-flight testing
Microservices
Availability
Failure resilient
Rolling updates
Recovery from failures
Azure Service Fabric
VNext PaaS offering
Battle tested – internally used by +5 years in large systems
State of the art orchestration and management
Stateful and Stateless programming model
271 CustomersService Fabric Preview
Azure Service Fabric + Cassandra
Massively scalable system
Cloud powered
Modern architecture patterns
Stateless Services
Cassandra Cluster
Cassandra Deployment Options
Azure Marketplace Templates GitHub Templates
Demo 1 – Cassandra on Azure
Large and Distributed Challenges
Ops Management High Density
Resource Governance
Deployment and Packaging
Rolling updates
Scalability
Hyper Scale
Load balancing
Scale Out
Maintainability
Granular versioning
In-flight testing
Microservices
Availability
Failure resilient
Rolling updates
Recovery from failures
Achieving High Density with Azure Service Fabric
IaaS Approach 1 Cluster – 1 VM
Azure Service Fabric(Stateless, stateful or Actor services)
• 1 service instance per VM with uneven workloads
• Lower compute density
• Slow in deployment & upgrades
• Slower in scaling and disaster recovery
• Many microservices per VM
• High microservices density
• Fast deployment & upgrades
• Fast scaling microservices across the cluster
Placement Constraints
You control your topology
Web
Internal API
N1DMZ = False
N2DMZ = True
N4DMZ = True
N5DMZ = True
N6DMZ = False
N3DMZ = False
Constraints = “(DMZ == False)”
Constraints = “(DMZ == True)”
Demo 2 – The Sample Application
Large and Distributed Challenges
Ops Management High Density
Resource Governance
Deployment and Packaging
Rolling updates
Scalability
Hyper Scale
Load balancing
Scale Out
Maintainability
Granular versioning
In-flight testing
Microservices
Availability
Failure resilient
Rolling updates
Recovery from failures
FD2FD1
UD2
UD1
Nodes will go down.
Fault DomainsSame physical location, HW failures
Upgrade DomainsSW upgrades to the underlying compute resources
Web Web
API API
Rolling UpdatesUD 1
UD 2 UD 3
N1
N2 N3
N4
N5 N6
V1
V1
V1
V2V2V2
Large and Distributed Challenges
Ops Management High Density
Resource Governance
Deployment and Packaging
Rolling updates
Scalability
Hyper Scale
Load balancing
Scale Out
Maintainability
Granular versioning
In-flight testing
Microservices
Availability
Failure resilient
Rolling updates
Recovery from failures
Demo 3 – Handling Failures & Rolling Upgrade
Large and Distributed Challenges
Ops Management High Density
Resource Governance
Deployment and Packaging
Rolling updates
Scalability
Hyper Scale
Load balancing
Scale Out
Maintainability
Granular versioning
In-flight testing
Microservices
Availability
Failure resilient
Rolling updates
Recovery from failures
Scaling and Balancing
• Declarative Model – “I want the topology to look like this”
• Service Fabric does the work• Balances Resources
• Moves services when hot
• Respects constraints
• You provide guidance• These metrics are important
• CPU
• Memory
• Etc.
• Balancing Threshold
• Activity Threshold
Demo 4 – Scalability
Questions