© 2014 VMware Inc. All rights reserved.

Skipping OpenStack Releases(You Don’t) Gotta Catch ‘Em All

Karol Stepniewski, Sidharth Surana, Mark T. Voelker

Oct. 27, 2016

Skipping Releases…Why?

• Once upon a time (in 2011) there weren’t very good incentives to skip releases

– Early days = each subsequent release delivers tons of high-demand features and fixes

• In 2016: core features are much more established

– Much more difficult: upgrade patterns not as well established, few/no projects with N-1 compatibility, etc

• In 2016: backward compatibility for some projects, DB migrations, versioned objects, more real-world upgrade experience

2

This is not 2011.

• Running OpenStack today? Chances are, you’re not running the latest release.

– October 2016 User Survey: more clouds on Juno, Kilo, or Liberty than Mitaka.

– So if deployers aren’t closely following the upstream release dates, are they at least following the six-month release cadence?

– OpenStack now fits in a great diversity of organizations, industries, and use cases

– “Stay close to tip on master” doesn’t work for everyone

– Neither does “upgrade every six months”

3

• Do you need time to qualify hardware/software upgrades?

• Is your current version working well for you?

• Do you have other priorities besides your IaaS?

• Do you time your upgrades to coincide with hardware refresh cycles, maintenance freezes, shopping seasons, audits, or fiscal calendars?

• Are you aggressive feature adopters, or using primarily core functionality?

• Are you using a product that provides a longer security window than upstream?

4

What’s Your Organization Like?

5

Today’s OpenStack is evolved enough to work for more models of consumption—and that means more

upgrade strategies are possible, too.

Skipping Releases…Why Not?

• You’re rolling your own with a small team and are dependent on upstream for bugfixes (and not just security bugfixes)

• You need features in a new release

• You use projects that iterate API’s or behaviors and potentially disruptively to your users

• You like living on the bleeding edge

6

Skipping releases: Ok, it’s a thing. But…how?

7

Think About What an Upgrade Entails

• Deploying new Python bits (libraries too)

• DB schema migrations

• Potential removal of old API’s and introduction of new ones (ex: LBaaSv1 -> LBaaSv2, Identity v2 -> Identity v3)

• Potential addition of components due to project refactoring (e.g. Ceilometer -> Gnocchi + Aodh, Nova -> Nova + Cinder)

• Potential upgrade of underpinning components (DB’s, RabbitMQ, hypervisors, network, server hardware, etc)

• Potential changes to the deployment architecture (ex: from a 15 vmdeployment to now a 8 vm deployment)

• Testing it all

• Turning it on for end users

8

How We Do Upgrades: Blue-Green Upgrade Pattern

9

Load Balancer

• Allows hardware to be swapped• Allows new control plane to be tested before going live• Very fast rollback• Allows for root causing of problems since both planes can be kept in event of failure• Skipping releases? No problem.• Leverages existing deployment code• Doesn’t depend on n-1 or n-2 compatibility in control plane components• Eases addition of new components/decomposition since green plane is “just a new deploy”

Kilo Control Plane Mitaka Control Plane

Demo

Does it really work?

• We have customers who’ve successfully upgraded from Icehouse -> Kilo and Kilo -> Mitakawith no major issues.

• We’ve had a few customers that performed skip-release upgrades without even telling us and had no problems.

• Favorite quote from one of our SE’s: “BTW just did an upgrade [from Kilo to Mitaka]...while drinking a beer & watching the game! How an OpenStack upgrade should be!”

11

So what’s special with VIO

12

Control

Plane

Data

Plane

Zero downtime for your deployed workloads across upgrades!

* Image source: http://blogs.vmware.com/openstack/author/amrabdelrazik/

Gotchas and Things to Plan For

• Extra resources needed for deploying the green control plane• IPs, CPU, Memory, Disk, …

• Watch out for fast-revving API’s and features that disappear in the span of the releases you’re skipping

• We don’t have this problem very often b/c we stick to projects that adhere to the standard deprecation tag/policy and only skip one release at a time…so it’s important to actually plan out a strategy

• Backward-incompatible API revs (LBaaS v1 vs v2)

• These tend to be a problem for end users more than cloud admins

• Prepare your end users about coming changes

E.g. if you are switching from keystone v2 to v3, the users

need to make sure their automation scripts against the cloud can

handle the v3 APIs

13

Questions?

14

Thank YouCome see us in the marketplace!