project sherpa: how rightscale went all in on docker
TRANSCRIPT
PROJECT SHERPA:
HOW RIGHTSCALE WENT ALL IN
ON DOCKER
• Tim Miller, VP Engineering
• Ryan O’Leary, Senior Director of Product
• Mark Dotson, Principal Systems Administrator & Lead Sherpa
• Ryan Williamson, Senior Manager, Engineering
• Tony Spataro, Senior Systems Architect
Panelists
Docker Use is Exploding
It’s Not Just Dev
• Goals, Timeline, Process
• Results
• Demo
• What Went Well/Hard
• Q & A Panel
Agenda
• RightScale Product
Strategy
• RightScale Bank
Account
• Productivity Increase
• Technical Staff’s
Career
Project Sherpa is good for everyone!
• 1028 running instances
• 670 running “dynamic” apps would be containerized
• Dynamic apps are RightScale developed apps, potentially new versions
released weekly, 52 in total.
• 358 running “static” apps would not be migrated (no ROI)
• SQL databases, Cassandra rings, MongoDB clusters, Redis,
Memcached, RabbitMQ systems, Syslogs, HTTP load balancers,
Collectd aggregators, proxy servers, and VPN endpoints.
Base Camp
• Expecting a 5 week hike, but
experienced a 7 week one
• 90% of the Eng Team for 5 weeks,
10% weeks 6,7
• 100% of OPS for 6 weeks, 50%
week 7
• Paused feature releases during
that time
• Split services into 3 groups
• Implemented “Bay of
Containers”
The Climb
• Converted 49 of 52 services to containers
• Reduced instance count by 55%
• Translated to a weighted avg savings of 53%
• 44% savings in Production
• 63% savings in Staging
• 74% savings in Integration
• Faster Developer Workflow
• Devs & QA now using laptops for dev/test vs. cloud based env
• Seeing velocity increases due to this less complex, self-contained
setup
• Reworking all our automation tools
The Summit
• Time while partially containerized in staging/prod was painful
• OPS/Production Support had to have 3 run books
• Developers were blocked
• Hotfixes to production took more time (2-3x)
• Minimize time you spend in this transition.
• If you are touching legacy apps, be ready for a bunch of Tech Debt
requests. Initially took the stance of “No”, but did end up letting some go
through.
• Don’t underestimate the effort around reworking dev/test/release tools.
• Needed more training for devs on the new branching and build workflows
• Experienced performance issues with Docker & Ubuntu 14.04 as Docker
Engine/host.
• Moving to CentOS 7 will evaluate later versions of Ubuntu
Challenges Encountered/Lessons Learned
