OPENSTACK-OPS

1

ABOUT THESE TRAINING MATERIALS

2 . 1

TRAINING MATERIALS WRITTEN BY ALTER WAY CLOUDCONSULTING

ex Osones -

Copyright © 2014 - 2019 alter way CloudConsultingLicense: Sources: HTML/PDF:

Creative Commons BY-SA 4.0 license

https://cloud-consulting.alterway.fr

Creative Commons BY-SA 4.0https://github.com/Alterway/formations/

https://osones.com/formations/

2 . 2

INTRODUCTION

3 . 1

GOALS OF THE TRAINING: OPENSTACKDiscover OpenStack and use its different servicesKnow how the project works and its capabilitiesUnderstand the internals of each OpenStack componentBe able to make the right configuration choicesBe capable of manually deploying an OpenStack cloudproviding IaaSKnow the best practices for deploying OpenStackBe able to track down the cause of an error in OpenStackBe able how to react in front of a bug and know the fixprocess

3 . 2

REQUIREMENTSAdvanced sys admin skills for Linux such as Ubuntu, Red Hator Debian, including:

Package managementConfiguration files and services handlingLVM (Logical Volume Management) and filesystems

Notions:Virtualization: KVM (Kernel-based Virtual Machine), libvirtNetwork: iptables, namespacesSQL

Optional:Comfortable in a Python environment

3 . 3

OPENSTACK: THE PROJECT

4 . 1

OVERVIEW

4 . 2

HIGH LEVEL

Simple version

4 . 3

HISTORYStarted in 2010Goal: the Free Open Source Cloud Operating SystemMerge of two projects from Rackspace (Storage) and NASA(Compute)Free software distributed under Apache 2.0 licenseBirth of the Foundation in 2012

4 . 4

MISSION STATEMENTTo produce a ubiquitous Open Source Cloud Computing platform that is easy to use, simple to implement, interoperable between deployments, works well at all scales, and meets the needs of users and operators of both public and private clouds.

4 . 5

RELEASESAustin (2010.1)Bexar (2011.1), Cactus (2011.2), Diablo (2011.3)Essex (2012.1), Folsom (2012.2)Grizzly (2013.1), Havana (2013.2)Icehouse (2014.1), Juno (2014.2)Kilo (2015.1), Liberty (2015.2)Mitaka (2016.1), Newton (2016.2)Ocata (2017.1), Pike (2017.2)Queens (2018.1), RockyRocky (2018.2)Stein (2019.1), Train (2019.2)Early 2020: Ussuri

4 . 6

SOME OF THE SUPPORTERS/CONTRIBUTORS ...Editors: Red Hat, Suse, Canonical, Vmware, ...Hardware makers: IBM, HP, Dell, ...Hardware makers/network: Juniper, Cisco, ...Hardware makers/storage: NetApp, Hitachi, ...Also: NASA, Rackspace, Yahoo, OVH, Citrix, SAP, ...GoogleGoogle! (since July 2015)

https://www.openstack.org/foundation/companies/

4 . 7

... AND USERSAll the previously mentioned contributorsIn France: CloudwattCloudwatt and NumergyNumergyWikimediaCERNPaypalComcastBMWEtc. Not counting confidential deployments

https://www.openstack.org/user-stories/

4 . 8

THE DIFFERENT SUB-PROJECTS

OpenStack Compute - NovaOpenStack (Object) Storage - SwiftOpenStack Block Storage - CinderOpenStack Networking - NeutronOpenStack Image Service - GlanceOpenStack Identity Service -KeystoneOpenStack Dashboard - HorizonOpenStack Telemetry - CeilometerOpenStack Orchestration - Heat

https://www.openstack.org/software/project-navigator/

4 . 9

THE DIFFERENT SUB-PROJECTS (2)

But also:Bare metal (Ironic)Queue service (Zaqar)Database service (Trove)Data processing (Sahara)DNS service (Designate)Shared File Systems (Manila)Key management (Barbican)Container (Magnum)

OthersClient CLI and librariesOpenStack deployment toolsLibraries used by OpenStackTools used to developOpenStack 4 . 10

APISEach project supports its OpenStack APISome projects support the corresponding AWS API(Nova/EC2, Swift/S3)

4 . 11

THE 4 OPENSOpen SourceOpen DesignOpen DevelopmentOpen Community

https://governance.openstack.org/tc/reference/opens.html

https://www.openstack.org/four-opens/

4 . 12

THE OPENSTACK FOUNDATIONMain governance entity and legal representation of theprojectBoard members are part of the sponsoring companies andelected by individual membersEveryone can (freely) become an individual memberHuman resources: marketing, event managemement, releasemanagement, a few developers (mainly on infrastructure)600 organizations across the world80000 individual members in 170 countries

4 . 13

THE OPENSTACK FOUNDATION

Main entities of the Foundation

4 . 14

OPEN INFRASTRUCTURELately, the OpenStack Foundation expands to OpenOpenInfrastructureInfrastructureBeyond OpenStack, new projects:

Kata ContainersZuulAirshipStarlingX

4 . 15

RESOURCESAnnouncements (new versions, security advisories):

Documentation portal: API/SDK: Project governance: Releases: Support:

openstack-discuss@lists.openstack.org#openstack@Freenode

openstack-announce@lists.openstack.orghttps://docs.openstack.org/

https://developer.openstack.org/https://governance.openstack.org/

https://releases.openstack.org/

https://ask.openstack.org/

4 . 16

RESOURCESNews:

Official blog: Planet: Superuser:

Commercial resources: among others

Job board:

https://www.openstack.org/blog/http://planet.openstack.org/

http://superuser.openstack.org/

https://www.openstack.org/marketplace/https://www.openstack.org/community/jobs/

4 . 17

USER SURVEYRegular survey done by the Foundation (every 6 months)Targets deployers and usersUsable data: https://www.openstack.org/analytics

4 . 18

CERTIFIED OPENSTACK ADMINISTRATOR (COA)The only certification:

Approved by the OpenStack FoundationNot linked to a specific company

Content:Mainly OpenStack cloud user oriented

Practical aspects:Practical exam, remote, duration: 2.5 hoursCost: $300 (one re-take possible)

Ressources

Tips: Handbook: (unofficial) Exercises:

https://www.openstack.org/coa/requirements/

https://www.openstack.org/coa/https://www.openstack.org/coa/tips/

http://www.openstack.org/coa/handbookhttps://github.com/AJNOURI/COA

4 . 19

RESOURCES - FRENCH COMMUNITY AND ASSOCIATION

Logo OpenStack-fr -

Meetups: Paris, Lyon, Toulouse, Montréal, etc.OpenStack Days France (Paris):

Attending events such as Paris Open Source SummitCommunication channels:

openstack-fr@lists.openstack.org#openstack-fr@Freenode

https://openstack.fr/ https://asso.openstack.fr/

https://openstackdayfrance.fr

4 . 20

INTERNALS

4 . 21

ARCHITECTURE

Detailed view of the services

4 . 22

IMPLEMENTATIONEverything is written in Python (Django for the web part)Each project is split in multiple services (example: API,scheduler, etc.)Re-use of existing components and existing librariesUsage of oslo.* libraries (developed by and for OpenStack):logs, config, etc.Usage of rootwrap to call underlying programs as root

4 . 23

IMPLEMENTATION - DEPENDENCIESDatabase: relational SQL (MySQL/MariaDB)Communication between services: AMQP(RabbitMQ)Caching: MemcachedDistributed storage of configuration (to come): etcd

4 . 24

DEVELOPMENT MODEL

4 . 25

STATS (2017)2344 developers65823 changes(commits)

https://www.openstack.org/assets/reports/OpenStack-AnnualReport2017.pdf

4 . 26

DEVELOPMENT: IN DETAILSOpen to all (individuals and companies)6 months release cycleEach cycle starts with a Project Team Gathering(PTG)During each cycle, an OpenStack Summit takes place

4 . 27

TOOLS AND COMMUNICATIONCode: Git (GitHub is used as a mirror)Peer review for code: GerritContinous Integration (CI): ZuulBlueprints/specifications and bugs:LaunchpadStoryBoardCommunication: IRC and mailing-listsTranslation: Zanata

4 . 28

DEVELOPMENT: IN DETAILS

Change workflow in OpenStack

4 . 29

RELEASE CYLE: 6 MONTHSThe schedule is published, example:

Milestone releasesFreezes: Feature, Requirements, StringRC releasesStable releasesSpecial case for some projects:

https://releases.openstack.org/stein/schedule.html

https://releases.openstack.org/reference/release_models.html

4 . 30

PROJECTSProject Teams:

Each deliverable has its own versioning - Semantic versioninghttps://governance.openstack.org/reference/projects/index.html

https://releases.openstack.org/

4 . 31

WHO CONTRIBUTES?Active Technical Contributor (ATC)Person with at least one recent contribution in a recognizedOpenStack projectVoting rights (TC and PTL)Core reviewer: ATC with permissions to approve patches in aprojectProject Team Lead (PTL): elected by the ATC of each projectStackalytics provides stats on contributionshttp://stackalytics.com/

4 . 32

WHERE TO FIND INFORMATIONS ABOUT THE OPENSTACKDEVELOPMENT

How to contribute

Various informations, on the wiki

Blueprints and bugs on Launchpad/StoryBoard

https://docs.openstack.org/project-team-guide/https://docs.openstack.org/infra/manual/

https://wiki.openstack.org/

https://launchpad.net/openstack/https://storyboard.openstack.org/https://specs.openstack.org/

4 . 33

WHERE TO FIND INFORMATIONS ABOUT THE OPENSTACKDEVELOPMENT

Proposed patches and their reviews on Gerrit

CI state (among others)

Code (Git) and tarballs are available

IRCFreenode networkLogs and meetings informations:

Mailing-lists

https://review.openstack.org/

http://status.openstack.org/

https://git.openstack.org/https://tarballs.openstack.org/

http://eavesdrop.openstack.org/

http://lists.openstack.org/

4 . 34

UPSTREAM TRAINING2 days trainingLearn how to become an OpenStackcontributorToolsProcessesWork and collaborate in an open way

4 . 35

OPENSTACK INFRATeam in charge of the OpenStack development infrastructureWorks like the OpenStack developement teams and uses thesame toolsResult: Infrastructure as code open sourceopen source

Uses (hybrid) cloudDevelops some tools:Zuulyaml2ical

https://opensourceinfra.org/

4 . 36

OPENSTACK SUMMITEvery 6 months at the middle of the development cycleIn the USA until 2013, now between North America andAsia/EuropeA few dozens at the beginning to thousands attendees todayAt the same time: conference (users, decision makers)andForum (developers/operators, replaces part of the previousDesign Summit)Defines the name of the next release: place/city near theSummit

4 . 37

EXAMPLE OF THE APRIL 2013 SUMMIT IN PORTLAND

4 . 38

EXAMPLE OF THE OCTOBER 2015 SUMMIT IN TOKYO

Photo: Elizabeth K. Joseph, CC BY 2.0, Flickr/pleia2

4 . 39

EXAMPLE OF THE OCTOBER 2015 SUMMIT IN TOKYO

Photo: Elizabeth K. Joseph, CC BY 2.0, Flickr/pleia2

4 . 40

EXAMPLE OF THE OCTOBER 2015 SUMMIT IN TOKYO

Photo: Elizabeth K. Joseph, CC BY 2.0, Flickr/pleia2

4 . 41

EXAMPLE OF THE OCTOBER 2015 SUMMIT IN TOKYO

4 . 42

PROJECT TEAM GATHERING (PTG)Since 2017At the beginning of each cycleReplaces part of the previous DesignSummitDedicated to developers

4 . 43

TRANSLATIONOfficial i18n teamOnly some parts are translated, like HorizonThe French translation is one of the most complete todayUse of a web platform based on Zanata:https://translate.openstack.org/

4 . 44

DEVSTACK: QUICKLY RUN OPENSTACK

4 . 45

DEVSTACK USE CASESQuickly deploy OpenStackUsed by developers to test their changesUsed for demosUsed to the the APIs without bothering about adeploymentMust NOT be used for production

4 . 46

DEVSTACK INTERNALSSupport for Ubuntu 16.04/17.04, Fedora 24/25, CentOS/RHEL7, Debian, OpenSUSEA shell script is responsible for everything: stack.shA config file: local.confInstalls all the required dependencies (packages)Clones all the git repositories (master branch by défaut)Launches all the components

4 . 47

CONFIGURATION: LOCAL.CONFExample

[[local|localrc]]ADMIN_PASSWORD=secreteDATABASE_PASSWORD=$ADMIN_PASSWORDRABBIT_PASSWORD=$ADMIN_PASSWORDSERVICE_PASSWORD=$ADMIN_PASSWORDSERVICE_TOKEN=a682f596-76f3-11e3-b3b2-e716f9080d50#FIXED_RANGE=172.31.1.0/24#FLOATING_RANGE=192.168.20.0/25#HOST_IP=10.3.4.5

4 . 48

USAGE TIPSDevStack installs a lot on the machineIt is recommended to work inside a VMTo test all the OpenStack components in good conditions,multiple Go of RAM are necessaryUse of Vagrant is recommended

4 . 49

DEPLOY OPENSTACK

5 . 1

WHAT WE ARE GOING TO SEEInstall OpenStack manually

Understand its internalsGo through each component in more detailsOverview of deployment solutions

https://docs.openstack.org/install-guide/

5 . 2

DETAILED ARCHITECTURE

Vue détaillée des services5 . 3

SERVICES ARCHITECTURE

5 . 4

A FEW GLOBAL CONFIGURATION ITEMSAll the components must be configured to talk with KeystoneMost must be configured to talk with MySQL/MariaDB andRabbitMQComponents split in multiple services sometimes have oneconfiguration file per serviceThe policy.json configuraton file specify the requiredpermissions for each API call

5 . 5

OPERATING SYSTEMLinux OS with PythonUbuntuRed HatSUSEDebian, Fedora, CentOS,etc.

5 . 6

PYTHON

Python logoOpenStack is Python 2.7 compatiblePython 3 comptability almost completeSo as not to reinvent the wheel, a lot of dependencies arenecessary

5 . 7

MYSQL/MARIADB DATABASEStores most of the data managed by OpenStackEach component has it own databaseOpenStack uses the SQLAlchemy Python ORMTheoretical support of what SQLAlchemy supports (andmigrations support)MySQL/MariaDB is the most tested and used implementationSQLite is mainly used for tests and demosSome deployments work with PostgreSQL

5 . 8

MESSAGE BUSAMQP: Advanced Message Queuing ProtocolMessages, queues, routingOpenStack processes interact through AMQPMultiple possible implementations: Qpid, 0MQ,etc.RabbitMQ by default

5 . 9

RABBITMQ

RabbitMQ logoRabbitMQ is written in ErlangAn Erlang virtual machine is thereforerequired

5 . 10

“HELLO WORLD” RABBITMQ

Simple example of RabbitMQ operation

5 . 11

MEMCACHED CACHEMultiple services make use of a cachingmechanismMemcached is the default implementation

5 . 12

KEYSTONE: AUTHENTICATION, AUTHORIZATION AND SERVICECATALOG

5 . 13

INSTALL AND CONFIGURATIONAPT package: keystoneWSGI web server integration (Apache by default)Configuration file: /etc/keystone/keystone.confUsers/groups backends: SQL, LDAP (or Active Directory)Projects/roles/services/endpoints backends: SQLTokens backends: SQL, Memcache, none (depending on thetoken type)

5 . 14

TOKENS DRIVERSUuidPKIFernet

5 . 15

BOOTSTRAPServices and endpoints creation (starting withKeystone)Users, groups and domains creationBootstrap feature

5 . 16

NOVA: COMPUTE

5 . 17

NOVA APITwo jobsAPI to manage instances for the utilisateurAPI for the instances: metadata APIThe metadata API must be available athttp://169.254.169.254/The metadata API provides personalized configurationinformations to each instance

5 . 18

NOVA COMPUTEManages instances (virtual or bare metal machines)Takes advantage of libvirt or other APIs such as XenAPIDrivers: libvirt (KVM, LXC, etc.), XenAPI, VMWare vSphere,IronicAbility to retrieve console logs and VNC console

5 . 19

NOVA SCHEDULERService in charge of scheduling instance requests to computenodesFilter, Chance, Multi SchedulerFilters, by default:AvailabilityZoneFilter,RamFilter,ComputeFilterSort by weigh, by default: RamWeigher

5 . 20

NOVA SCHEDULER IN ACTION

nova-scheduler operation

5 . 21

NOVA CONDUCTOROptional service which improves securityAct as a proxy between compute nodes and DBCompute nodes, at risk, therefore don't have DB accessanymore

5 . 22

GLANCE: IMAGE REGISTRY

5 . 23

BACKENDSSwift or S3CephHTTPLocaldirectory

5 . 24

INSTALLAPT package: glance-api

5 . 25

NEUTRON: NETWORK AS A SERVICE

5 . 26

PRINCIPLESSoftware Defined Networking (SDN)Was Quantum and nova-networkneutron-server: provides the APIDHCP agent: provides DHCP service to instancesL3 agent: manages network layer 3, routingPlugin: LinuxBridge by default, other open source /proprietary, software/hardware implementations exist

5 . 27

ADDITIONAL FEATURESBeyonce basic layer 2 and 3 networking features, Neutron can

provide other services:

Load Balancing (HAProxy, ...)Firewall (vArmour, ...): different from security groupsVPN (Openswan, ...): to reach a private network withoutfloating IPs

These features are also based on plugins

5 . 28

PLUGINS ML2Modular Layer 2Modular Layer 2LinuxBridgeOpenVSwitchOpenDaylightContrail, OpenContrailNuage NetworksVMWare NSXcf. OpenFlow

5 . 29

IMPLEMENTATIONEach network is a bridgeBridges are expanded across hosts using tunnels (VXLANtype) if necessaryNeutron takes advantage of Linux kernel networknamespaces to allow IP overlappingMetadata proxy is a component that allows instances isolatedin their network to reach the metadata API provided by Nova

5 . 30

DIAGRAM

User view of the network

5 . 31

DIAGRAM

5 . 32

CINDER: BLOCK STORAGE

5 . 33

PRINCIPLESWas nova-volumeProvides volumesVolume attachement through iSCSI bydefault

5 . 34

INSTALLPackage cinder-api: provides the APIPackage cinder-volume: creation and management ofvolumesPackage cinder-scheduler: scheduling of volume creationrequestsPackage cinder-backup (optional): backup to an object store

5 . 35

BACKENDSUse of multiple backends in parallelpossibleLVM (by default)GlusterFSCephProprietary storage systems such as NetAppDRBD

5 . 36

HORIZON: WEB DASHBOARD

5 . 37

PRINCIPLESHorizon is a Django moduleOpenStack Dashboard is the official implementation of thismodule

Python Django web framework logo5 . 38

CONFIGURATIONlocal_settings.pyServices appear in Horizon if they exist in Keystone servicecatalog

5 . 39

SWIFT: OBJECT STORAGE

5 . 40

PRINCIPLESSDS: Software Defined StorageUse of commodity hardwareCAP theorem: choosing twoCompletly decentralized architectureNo central database

5 . 41

IMPLEMENTATIONProxy: API server for all the requestsObject server: storage serverContainer server: maintains list of objects incontainersAccount server: maintains list of containers inaccountsEach object is replicated n times (3 by default)

5 . 42

THE RINGProblem: how to decide which data is going onto whichobject serverThe ring is split in partitionsEach data is located in the ring to find out the partitionA partition is associated to multiple servers

5 . 43

DIAGRAM

Swift architecture5 . 44

CEILOMETER: METRICS COLLECTION

5 . 45

MONITOR USAGE OF INFRASTRUCTURE WITH CEILOMETERStores different metrics regarding usage of cloudservicesProvides APIs to retrieve these dataBase to build billing tools (example: CloudKitty)

5 . 46

CEILOMETERRetrieves data and storesthemUsed to be stored in MongoDBNow stored in Gnocchi

5 . 47

GNOCCHI: TIME-SERIES DATABASEWhy Gnocchi? To solve Ceilometer + MongoDB scaling issuesInitiated by Ceilometer developers and integrated in theCeilometer project teamProvides an API to read and write dataUses a relational DB and an object storage system

5 . 48

HEAT: RESOURCES ORCHESTRATION

5 . 49

ARCHITECTUREheat-apiheat-engine

5 . 50

SOME OTHER INTERESTING COMPONENTS

5 . 51

TROVE: DATABASE AS A SERVICEtrove-api: APItrove-taskmanager: manages DB instancestrove-guestagent: internal agent in instances

5 . 52

DESIGNATE: DNS AS A SERVICEManages different backends: BIND, PowerDNS,etc.

5 . 53

BARBICAN: KEY MANAGEMENT AS A SERVICEPossible backends:

Encrypted filesPKCS#11KMIPDogtag

5 . 54

MAGNUM: CONTAINER INFRASTRUCTURE AS A SERVICEBackends: Swarm,Kubernetes

5 . 55

OPENSTACK IN PRODUCTION AND OPERATIONS

6 . 1

BEST PRACTICES FOR A PRODUCTION DEPLOYMENT

6 . 2

WHICH COMPONENTS SHOUD I INSTALL?Keystone is mandatoryUse of Nova goes with Glance and NeutronCinder and Swift usefulness depends on storage needsSwift can be used separately from other componentsHeat doesn't cost muchHigher level services need to be evaluated case bycase

https://docs.openstack.org/arch-design/

6 . 3

THINK ABOUT FUNDAMENTAL CHOICES AT THE BEGINNINGDistribution and deployment methodUpdate and upgrade policyDrivers/backends: hypervisor, block storage,etc.Network: what architecture and what drivers

6 . 4

DIFFERENT INSTALLATION METHODSForget about DevStack for productionManual deployment as seen previously is not recommendedby unmaintainablePackaged and ready to use OpenStack distributionsClassical distributions and configuration managementContinuous deployment

6 . 5

MAJOR UPGRADESOpenStack supports N → N+1 upgradesSwift: very good rolling upgrade supportOther components: test with you data firstRead release notesCf. CERN blog posts https://techblog.web.cern.ch/techblog/

6 . 6

STABLE UPDATESMajor bug and security fixes are providedOpenStack includes these fixes as patches in the stablebranchPoint releases are published and includes fixes from thestable branchStable version support timeframe is variable, depending onintegrators' interest

6 . 7

ASSIGN ROLES TO MACHINESLots of documentations mention these roles:

Controller node: APIs, DB, AMQPNetwork node: DHCP, router, floating IPsCompute node: Hypervisor/instancesmanagement

This simplified model is not HA.

6 . 8

HIGH AVAILABILITYIaaS High Availability

MySQL/MariaDB, RabbitMQ: classical HA (Galera, Clustering)API services Are stateless and HTTP: scale out and loadbalancersMost other OpenStack services are able to scale out as well

HA guide:

Talks by Florian Haas, Hastexo:

https://docs.openstack.org/ha-guide/

https://www.openstack.org/community/speakers/profile/398/florian-haas

6 . 9

HIGH AVAILABILITY OF THE NEUTRON L3 AGENTDistributed Virtual Router (DVR)L3 agent HA (VRRP)

6 . 10

APIS CONCERNSUniform URLs for all the APIs:

Use a reverse proxyDon't forget to update the service catalog

Apache/mod_wsgi to serve APIs when possible (Keystone,etc.)

Operations guide: https://docs.openstack.org/openstack-ops/content/

6 . 11

NETWORKSManagement network: administrative networkData/instances network: network for inter instances trafficExternal network: external network, in the existing networkinfrastructureStorage network: network for Cinder/Swift storageAPI network: network containing API endpoints

6 . 12

SECURITY RELATED CONCERNSMust-have: HTTPS for external API accessSecuring MySQL/MariaDB and RabbitMQ trafficOne MySQL/MariaDB access per database and per serviceOne Keystone user per serviceLimit read permissions to configuration files (passwords,token)Security vulnerabilities: OSSA (OpenStack Security Advisory),OSSN (... Notes)

Security guide: https://docs.openstack.org/security-guide/

6 . 13

SEGMENT A CLOUDHost aggregates: physical hosts with similar featuresAvailability zones: hosts depending on the same electricalsupply, same switch, same DC, etc.Regions: each region has its own APICells: gather multiple clouds within a unique API

https://docs.openstack.org/openstack-ops/content/scaling.html#segregate_cloud

6 . 14

HOST AGGREGATESNova specificAdmin defines host aggregates through the APIAdmin associates flavors and aggregates through commonkey/values1 aggregate ≡ 1 similarity, ex: GPUUser chooses an aggregate through their flavor choice whencreating an instance

6 . 15

AVAILABILITY ZONESNova and Cinder specificHosts groupsSplit in terms of availability: Rack, Datacenter, etc.User chooses an availability zone when creating an instanceUse can request instances to be started in the same zone, oron the contrary in different zones

6 . 16

REGIONSGeneric in OpenStackAWS regions counterpartA service can have different endpoints in differentregionsEach region is autonomousUse case: very large cloud (such as some public clouds)

6 . 17

CELLSNova specificOnly one nova-api in front of mutiplecellsEach cell has its own DB and message busAdds a scheduling layer (cell choice)

6 . 18

OPENSTACK PACKAGING - UBUNTUPackaging is done in multiples distributions, RPM, DEB andothersUbuntu historically is the reference platform for OpenStackdevelopementPackaging in Ubuntu closely follows OpenStackdevelopment, and automated tests are performedCanonical provides the Ubuntu Cloud Archive, which includesthe latest OpenStack version for the latest Ubuntu LTS

6 . 19

UBUNTU CLOUD ARCHIVE (UCA)

OpenStack support in Ubuntu through UCA6 . 20

OPENSTACK PACKAGING IN OTHER DISTRIBUTIONSOpenStack is integrated in Debian's official repositoryRed Hat provides RHOS/RDO (deployment based on TripleO)Like Ubuntu, Fedora's release cycle is synchronized withOpenStack's

6 . 21

OPENSTACK DISTRIBUTIONSStackOps: historyMirantis: FuelHP Helion: Ansiblecustometc.

6 . 22

TRIPLEOOpenStack On OpenStackGoal: ability to deploy an OpenStack cloud (overcloud) froman OpenStack cloud (undercloud)Autoscaling of the cloud itself: deployment of new computenodes when necessaryWorks jointly with Ironic for bare metal deployment

6 . 23

BARE METAL DEPLOYMENTOpenStack bare metal hosts deployment can be managedwith the help of dedicated toolsMaaS (Metal as a Service), by Ubuntu/Canonical: works withJujuCrowbar / OpenCrowbar (initially Dell): uses ChefeDeploy (eNovance): image based deploymentIronic through TripleO

6 . 24

TEMPESTTest suite of an OpenStack cloudMakes API calls and checks the resultUsed a lot by developers through continuous integrationDeployers can use Tempest to check their cloud's complianceSee also Rally

6 . 25

CONFIGURATION MANAGEMENTPuppet, Chef, CFEngine, Saltstack, Ansible, etc.These tools can help deploying an OpenStackcloud... but also to manage instances (next section)

6 . 26

MODULES PUPPET, PLAYBOOKS ANSIBLEPuppet OpenStack and OpenStack Ansible: Puppet modulesand Ansible playbooksDeveloped as part of the OpenStack projecthttps://wiki.openstack.org/wiki/Puppethttps://docs.openstack.org/developer/openstack-ansible/install-guide/

6 . 27

CONTINUOUS DEPLOYMENTOpenStack maintains an always stable master (trunk)Possible to deploy master on a daily basis (CD: ContinousDelivery)Requires setting up an important infrastructureEases upgrades between major versions

6 . 28

FACING ISSUES

6 . 29

TIPS IN CASE OF ERROR OR FAULTY BEHAVIORAre we working on the appropriate project?Is the API responding with an error? (the dashboard may hidesome informations)If going further is required:

Look into logs on thje cloud controller(/var/log/<composant>/*.log)Look into logs on the compute node and the network nodeif the issue is network/instance specificMay change logs verbosity in the configuration

6 . 30

IS IT A BUG?If the CLI client crashes, it's a bugIf the web dashboard or the API responds with an error 500, itmight be a bugIf the logs show a Python stacktrace, it's a bugOtherwise, you decide

6 . 31

OPERATIONS

6 . 32

LOGS MANAGEMENTCentralize logsAPI logsOther OpenStack components logsDB, AMQP, etc. logs

6 . 33

BACKUPDatabasesDeployment mechanism, rather than configurationfiles

6 . 34

MONITORINGAPI responseChecking OpenStack services anddependencies

6 . 35

QUOTAS USAGELimit the number of allocableresourcesPer user or per tenantSupport in NovaSupport in CinderSupport in Neutron

6 . 36

CONCLUSION

7 . 1

TO CONCLUDECloud is a revolution for ITOpenStack is the open source flagship project for the IaaSpartDeploying OpenStack is not an easy taskUsing an IaaS cloud implies changes of practiceSoftware and infrastructure architecture jobs are changing

7 . 2