openstack ha

11
Openstack HA

Upload: deepak-mane

Post on 15-Jul-2015

66 views

Category:

Technology


0 download

TRANSCRIPT

Page 1: Openstack ha

Openstack HA

Page 2: Openstack ha
Page 3: Openstack ha

• OpenStack can be made highly available by installing redundant OpenStack services (Keystone, Nova, Glance etc.) on different machines

• let Pacemaker control execution of the OpenStackservices. Custom resource agents must be installed in order to allow the LRMDs to manage OpenStack resources.

• the CIB must be configured so the CRM can orchestrate the LRMD actions.

Page 4: Openstack ha

Why is Pacemaker slow?

Page 5: Openstack ha

Suboptimal initialization scripts

• OpenStack services do not generate a file containing the process identification (pid) in a pid file per default. Therefore Pacemaker is not able to identify OpenStack services as manageable entities or resources. Some hacking is necessary in order to make OpenStack services Pacemaker-compliant.

Page 6: Openstack ha

• Custom resource agents: there are no OCF-compliant OpenStack resource agents delivered out of the box. Pacemaker’s Local Resource Management Daemons (LRMDs) are therefore not able to manage OpenStackservices

Page 7: Openstack ha

• Bad Cluster Information Base (CIB) configuration: The worst thing is a messy CIB configuration. If e. g. recovery tasks are kept in large groups and monitoring intervals are too long to discover outages very fast, the Pacemaker recovery will act very slowly, because Pacemaker has to recover large resource groups and recovery actions are started lately.

Page 8: Openstack ha

Improvements

Page 9: Openstack ha

What can be done to make Pacemaker faster?

• Optimize initialization scripts: Depending on your initialization system (Init-V, Upstart, Systemd), you must customize the upstart of services in order to generate pidfiles which help Pacemaker to identify the service on the system. OpenStack services in Ubuntu are upstarted by the Init-V system. If you run OpenStack on Ubuntu, you must customize the upstart scripts so they will generate pid files automatically. This can be done by changing the configuration files in /etc/init. For the quantum server e. g. you have to change the /etc/init/quantum-server.conf file to contain several lines which tell the upstart daemon to create a pidfile and place it in a specified folder (typically /var/run). Creation of pid files can be performed using the start-stop-daemon

Page 10: Openstack ha

Create custom resource agents

• there are no OpenStack resource agents delivered out of the box, but you can create them if you want. Resource agents must be placed in the /usr/lib/ocf/resource.d/ folder. They must contain methods to monitor, start and stop services as well as a method to control the execution status of the service. Some good examples for OpenStack resource agents can be found on the Hastexo website

Page 11: Openstack ha

Improve Cluster Information Base (CIB) configuration

• Most improvements can be done by changing the CIB configuration. Ideally OpenStack services should run redundantly at the same time on two different OpenStack nodes which can be reached by using a shared virtual IP. In case of a service failure on one node, Pacemaker just has to route traffic to the node where the service is still running. If the service is not running redundantly on the fallback node before the failure occurs, Pacemaker has to upstart the service on at least one of the nodes.