Download - Implementing Docker in Production at Scale
1
Implementing Dockerin Production at Scale
Sean P. Kane & Karl MatthiasLead Site Reliability Engineers
2Confidential ©2008-15 New Relic, Inc. All rights reserved.
This document and the information herein (including any information that may be incorporated by reference) is provided for informational purposes only and should not be construed as an offer, commitment, promise or obligation on behalf of New Relic, Inc. (“New Relic”) to sell securities or deliver any product, material, code, functionality, or other feature. Any information provided hereby is proprietary to New Relic and may not be replicated or disclosed without New Relic’s express written permission.
Such information may contain forward-looking statements within the meaning of federal securities laws. Any statement that is not a historical fact or refers to expectations, projections, future plans, objectives, estimates, goals, or other characterizations of future events is a forward-looking statement. These forward-looking statements can often be identified as such because the context of the statement will include words such as “believes,” “anticipates,” “expects” or words of similar import.
Actual results may differ materially from those expressed in these forward-looking statements, which speak only as of the date hereof, and are subject to change at any time without notice. Existing and prospective investors, customers and other third parties transacting business with New Relic are cautioned not to place undue reliance on this forward-looking information. The achievement or success of the matters covered by such forward-looking statements are based on New Relic’s current assumptions, expectations, and beliefs and are subject to substantial risks, uncertainties, assumptions, and changes in circumstances that may cause the actual results, performance, or achievements to differ materially from those expressed or implied in any forward-looking statement. Further information on factors that could affect such forward-looking statements is included in the filings we make with the SEC from time to time. Copies of these documents may be obtained by visiting New Relic’s Investor Relations website at ir.newrelic.com or the SEC’s website at www.sec.gov.
New Relic assumes no obligation and does not intend to update these forward-looking statements, except as required by law. New Relic makes no warranties, expressed or implied, in this document or otherwise, with respect to the information provided.
3Confidential ©2008-15 New Relic, Inc. All rights reserved.
Overview - New Relic’s conversion to Docker containers
- Building an MVP platform
- Lessons we learned from 1.5 years in production
- Building a v.2 platform
4Confidential ©2008-15 New Relic, Inc. All rights reserved.
The Incentives
4
http://newrelic.com/insights
SOA initiative Monolithic to Micro-Services
New product launch Greener Fields Doubling our server count
Inconsistent deployment story Per team Capistrano & Rake
5Confidential ©2008-15 New Relic, Inc. All rights reserved.
A Simple Platform that Scaled Well…
Where Our Platform Got Us
5
5
250+ production containers
75+ project
s
15+ teams
6Confidential ©2008-15 New Relic, Inc. All rights reserved.
Simple approach…
Our First Platform
6
6
Solve Deployment First Simple Solutions Work. Avoid Big Design Up Front
7Confidential ©2008-15 New Relic, Inc. All rights reserved.
Our First Platform
7
7
Remote Logging
Static Load Balancing
Use dedicated ports, everywhereOur New
PlatformOld School
Service Discovery
8Confidential ©2008-15 New Relic, Inc. All rights reserved.
Our First Platform
8
8
Process Management
Supervisord in all the containers
9Confidential ©2008-15 New Relic, Inc. All rights reserved.
Our First Platform
9
9
Remote Logging
Supervisord, Heka, and Syslog
Build out your logging story up front
10Confidential ©2008-15 New Relic, Inc. All rights reserved.
Our First Platform
10
10
Nagios and cAdvisor
New Relic Server Monitor w/ Docker support
/status/check (200 OK)
Monitor overall service health versus containers.
Stats and Monitoring
11Confidential ©2008-15 New Relic, Inc. All rights reserved.
Our First Platform
11
11
Stats & Monitoring
New Relic Server Monitor w/Docker support
12Confidential ©2008-15 New Relic, Inc. All rights reserved.
Our First Platform
12
12
Application Standards
/status/checkExternalized ConfigurationStandardized base images for Docker
13Confidential ©2008-15 New Relic, Inc. All rights reserved.
Our First Platform
Shipright (Internal Tool)
Resource Management
Port Registry (Internal Tool)
Centurion
Service Discovery
Docker build tool
14Confidential ©2008-15 New Relic, Inc. All rights reserved.
Our First Platform
Shipright (Internal Tool)
Resource Management
Port Registry (Internal Tool)
Centurion
Old-school Service Discovery
Noisy neighbors - cgroups
A single, large Docker cluster is better than multiple smaller team-based clusters
15Confidential ©2008-15 New Relic, Inc. All rights reserved.
Our First Platform
Shipright (Internal Tool)
Resource Management
Port Registry (Internal Tool)
Centurion
Service Discovery
Centurion
Where do services live?
16Confidential ©2008-15 New Relic, Inc. All rights reserved.
Our First Platform
Shipright (Internal Tool)
Resource Management
Port Registry (Internal Tool)
Centurion
Service Discovery
Docker Deployment Tool for Repeatable Deployments
https://github.com/newrelic/centurion Rolling, zero-down-time
deploys
Any engineer can deploy any app
Configuration & secrets are not in the application repo
17Confidential ©2008-15 New Relic, Inc. All rights reserved.
Our New Platform
Mesos - Resource offers
Marathon – Scheduler
Custom Executor
Dogestry – S3 CLI image registry
https://github.com/dogestry/dogestry
18Confidential ©2008-15 New Relic, Inc. All rights reserved.
Service Discovery
Distributed Load Balancing
Health Check Monitoring
Intend to Open Source later in 2015
Tooling We’re Building
18
18
Sidecar
20Confidential ©2008-15 New Relic, Inc. All rights reserved.
The simple approach worked.
It provided us time to learn as an organization.
It allowed us to iterate fast and often.
It provided us the time to build something better. That we needed.
Conclusion
20
20
21Confidential ©2008-15 New Relic, Inc. All rights reserved.
Presenter Name, Title and or Date
Thanks
@relistan – Karl Matthias
@spkane – Sean P. Kane
http://goo.gl/AM943Mhttp://newrelic.com/docker