openstack service high availabilityopenstackpakistan.github.io/web/resources/meetup_2nd_ha.pdf ·...
Post on 18-Jun-2018
251 Views
Preview:
TRANSCRIPT
OpenStack Service High AvailabilityOpenstack Pakistan meetup, April 2016
Syed Affan Ahmed
Director Engineering, PLUMgrid
Copyright © PLUMgrid, Inc. 2011-2016
• OpenStack and its Services
• High Availability in Openstack: Overview
• Services HA
• Keystone
• Glance
• Cinder
• Nova
• HA deployment architecture (one type)
Agenda
High Availability in OpenStack
Copyright © PLUMgrid, Inc. 2011-2016
• Designed to minimize two aspects
• System downtime
• Data loss
Both in the context of single-point-of-failure and cascading set of events
High Availability Principles
Copyright © PLUMgrid, Inc. 2011-2016
• OpenStack Services HA (our focus now)• Architectural support
• Important if a public cloud offering is part of ROI
• Application HA• Cloud native (scalable)
• Legacy (pets) • Similar to Services HA
• Load-balanced
• Shared storage and hypervisor-based
Types of HA
Copyright © PLUMgrid, Inc. 2011-2016
• Stateless
• Provides a response to each individual request
• nova-api, nova-conductor, glance-api, keystone-api, neutron-api and nova-
scheduler
• HA achieved via redundant instances & load-balancer
• Stateful
• Provides a response to a request that is based on previous requests
• OpenStack databases & message queue
• HA requires more substantial configuration
Stateless vs Stateful Servicesand the impact on complexity
Copyright © PLUMgrid, Inc. 2011-2016
• Active/Passive brings additional resources online to replace those
that have failed
• Single master
• Active/Active has all resource being used concurrently
• Multi-master
A/P and A/A HA ConfigurationDesign choices
Openstack Services HA An illustrative example
Copyright © PLUMgrid, Inc. 2011-2016
• Pacemaker • Cluster management
• Virtual IP (can also be done with keepalived)
• STONITH
• HAProxy• Load balancing and service failure detection
Two key components
Copyright © PLUMgrid, Inc. 2011-2016
Give Me Horizon
Web UI Now!
Horizon Controller
HA illustratedFullsuit edition!
Inspired by Arthur Berezin’s talk : “Deep Dive into Highly Available OpenStack Architecture”
Copyright © PLUMgrid, Inc. 2011-2016
Give Me Horizon
Web UI Now!
Horizon Controller
Single Point of Failure
Copyright © PLUMgrid, Inc. 2011-2016
Give Me Horizon
Web UI Now!
HAProxy Controller 1
Horizon Controller 1 Horizon Controller 1 Horizon Controller 1
Copyright © PLUMgrid, Inc. 2011-2016
Give Me Horizon
Web UI Now!
HAProxy Controller 1
Horizon Controller 1 Horizon Controller 2 Horizon Controller 3
Single Point of Failure
Each Could Fail
Copyright © PLUMgrid, Inc. 2011-2016
Give Me Horizon
Web UI Now!
HAProxy Controller 1
Horizon Controller 1 Horizon Controller 2 Horizon Controller 3
Single Point of Failure
Pacemaker Cloned Horizon Service
Copyright © PLUMgrid, Inc. 2011-2016
Give Me Horizon
Web UI Now!
Horizon Controller 1 Horizon Controller 2 Horizon Controller 3
Pacemaker Cloned Horizon Service
HAProxy Controller 1 HAProxy Controller 2 HAProxy Controller 3
Pacemaker Cloned HAProxy Service
Copyright © PLUMgrid, Inc. 2011-2016
Give Me Horizon
Web UI Now!
Horizon Controller 1 Horizon Controller 2 Horizon Controller 3
Pacemaker Cloned Horizon Service
HAProxy Controller 1 HAProxy Controller 2 HAProxy Controller 3
Pacemaker Cloned HAProxy ServiceHorizon
VIP
Services HA
All diagrams inspired from Arthur’s talk!
Copyright © PLUMgrid, Inc. 2011-2016
• Cloned Stateless HTTPD Service
• Same SSL Certs on all nodes
• Cache is local on each host
Keystone HA
ClonedHAProxy HAProxy
ClonedHTTPd/
Keystone
HTTPd/
Keystone
Keystone
VIP
pcsd
Node 1
pcsd
Node 2
STONITH STONITH
Copyright © PLUMgrid, Inc. 2011-2016
•Both services (API and Registry) are Cloned Active/Active • Both are load balanced
and VIP-ed
• Active VIP on different nodes
Glance HA
ClonedHAProxy HAProxy
ClonedGlance
Registry
Glance
Registry
Glance
API
VIP
pcsd
Node 1
pcsd
Node 2
STONITH STONITH
Glance
Registry
VIP
ClonedGlance-API Glance-API
Copyright © PLUMgrid, Inc. 2011-2016
•Cinder-API, scheduler are Stateless•Cloned, LB and VIP
•Cinder-Volume is A/P due to potential race conditions
Cinder HA
Storage
ClonedHAProxy HAProxy
Cinder
API
VIP
pcsd
Node
1
pcsd
Node
2STONITH STONITH
ClonedCinder-API Cinder-API
ClonedScheduler Scheduler
A/PVolume VolumeDriver Driver
Copyright © PLUMgrid, Inc. 2011-2016
• Nova-API configured with LB and VIP
• Nova-API, Nova-Scheduler and Nova-Conductor are Stateless A/A cloned services
Nova Controller HA
ClonedHAProxy HAProxy
ClonedConductor Conductor
Nova
API
VIP
pcsd
Node
1
pcsd
Node
2STONITH STONITH
ClonedNova-API Nova-API
ClonedScheduler Scheduler
Copyright © PLUMgrid, Inc. 2011-2016
• Provider specific, and typically a SDN solution
• PLUMgrid most scalable solution out there• Fully replicated Control Plane and
• a fully distributed Data Plane
Neutron HA
Copyright © PLUMgrid, Inc. 2011-2016
A full Services HA deploymentRed Hat reference Architecture
Copyright © PLUMgrid, Inc. 2011-2016
A full DCN deployment the above HA Architecture
(408) 800-7586 www.plumgrid.com
5155 Old Ironsides Dr.
Suite 200
Santa Clara, CA 95054,THANK YOU!Keep in Touch and Contact Us
top related