linux tag 2013 data center automation with yadt
DESCRIPTION
In this talk we will talk about the design decisions behind YADT and how it helps reduce the complexity of your datacenter infrastructrure while at the same time enabling you to fully automate almost all of it. YADT - an Augmented Deployment Tool is a next generation data center deployment and system management tool. It consists of two central components: Firstly, a central management of dependencies between services, systems and software packages, and secondly a deployment based on packages for both software and configuration management. YADT is developed at and used by ImmobilienScout24 as an Open Source project. The software makes it easy to deploy your software, configuration and operating system updates. It also provides a service-oriented abstraction layer which makes dependencies explicit and comprehensible. YADT will update packages and restart the services in the correct order within your data center. It understands the traditional dependencies between packages as well as dependencies between services, even over the network.TRANSCRIPT
Data Center Automationwith YADT
Berlin | 23.05.2013 | Schlomo SchapiroSystems Architect, Open Source Evangelist
License: http://creativecommons.org/licenses/by-nc-nd/3.0/
Slide 2 | Data Center Automation with YADT | Schlomo Schapiro
www.ImmobilienScout24.de
>2 billion PI per month
2 data center with ~1400 VM
total of ~600 employees
~30 crossfunctional IT teams
~160 in IT
15 years in business
part of Deutsche Telekom
Slide 3 | Data Center Automation with YADT | Schlomo Schapiro
Slide 4 | Data Center Automation with YADT | Schlomo Schapiro
Systems Management with YADT
Slide 5 | Data Center Automation with YADT | Schlomo Schapiro
The Big Picture – Continuous Delivery Platform
BusinessDecisionTo GoLive
All Humans are on the Same Side
BuildConfigDeployTestSystems-Management
Automation
Scale OutData Centers
Slide 6 | Data Center Automation with YADT | Schlomo Schapiro
1st task: Reduce Systemic Complexity
File Management
Systems Management
Slide 7 | Data Center Automation with YADT | Schlomo Schapiro
File Management with RPM & YUM
Slide 8 | Data Center Automation with YADT | Schlomo Schapiro
File Management with RPM & YUM
Libraries
Applications
Configuration
Operating System
100 %
Slide 9 | Data Center Automation with YADT | Schlomo Schapiro
File Management with RPM & YUM
PROD
DEV
InfrastructureBuild
ApplicationBuild
ConfigurationBuild
QA
YUMRepos
Slide 10 | Data Center Automation with YADT | Schlomo Schapiro
File Management with RPM & YUM
Interface
PROD
DEV
InfrastructureBuild
ApplicationBuild
ConfigurationBuild
QA
YUMRepos
Slide 11 | Data Center Automation with YADT | Schlomo Schapiro
PROD
DEV
InfrastructureBuild
ApplicationBuild QA
YUMRepos
Configuration Build – Pre-built Config Content
ConfigurationBuild
Slide 12 | Data Center Automation with YADT | Schlomo Schapiro
Configuration Build – Pre-built Config Content
SVN
commitconfig file
buildconfig RPM
RPMswithstatic
content
YUMRepo
Slide 13 | Data Center Automation with YADT | Schlomo Schapiro
TemplatingVariables
Overlaying
Configuration Build – Pre-built Config Content
SVN
commitconfig file
buildconfig RPM
RPMswithstatic
content
YUMRepo
Slide 14 | Data Center Automation with YADT | Schlomo Schapiro
we use1 RPM
per host
TemplatingVariables
Overlaying
Configuration Build – Pre-built Config Content
SVN
commitconfig file
buildconfig RPM
RPMswithstatic
content
YUMRepo
Slide 15 | Data Center Automation with YADT | Schlomo Schapiro
Sharedresponsability
Ops, Dev, QA ...
we use1 RPM
per host
TemplatingVariables
Overlaying
Configuration Build – Pre-built Config Content
SVN
commitconfig file
buildconfig RPM
RPMswithstatic
content
YUMRepo
Slide 16 | Data Center Automation with YADT | Schlomo Schapiro
Application Build – Continuous Release Delivery
PROD
DEV
InfrastructureBuild
QA
YUMRepos
ConfigurationBuild
ApplicationBuild
Slide 17 | Data Center Automation with YADT | Schlomo Schapiro
Application Build – Package Promotion Pipeline
YUMRepo
TestedRepo
YUMRepo
ReleasedRepo
YUMRepo
ComittedRepo
DEV Pre-PROD PROD
Slide 18 | Data Center Automation with YADT | Schlomo Schapiro
Application Build – Package Promotion Pipeline
YUMRepo
TestedRepo
YUMRepo
ReleasedRepo
YUMRepo
ComittedRepo
DEV Pre-PROD PROD
CI
Deploy&
Test
Auto-promote release if tests OK
Deploy&
Test
Business decision
to Go Live with
tested versions
Deploy&
Test
Slide 19 | Data Center Automation with YADT | Schlomo Schapiro
File Management through many YUM repositories
PROD
DEV
InfrastructureBuild
ApplicationBuild
ConfigurationBuild
QAYUMRepo
YUMRepoYUM
RepoYUMRepo
Slide 20 | Data Center Automation with YADT | Schlomo Schapiro
Systems Management with YADT
Slide 21 | Data Center Automation with YADT | Schlomo Schapiro
YADT – Unified Dependency Tree
Web Server
HTTPD
Tomcat
App Server
Load Balancer
Slide 22 | Data Center Automation with YADT | Schlomo Schapiro
HTTPD
Tomcat
web-app
httpd
config
tomcat
YADT – Unified Dependency Tree
initscript
initscript
Slide 23 | Data Center Automation with YADT | Schlomo Schapiro
YADT – Large Scale System Management
Web Server
HTTPD
Tomcatweb-app
httpd
config
tomcat
App Server
Java App
app
config
Slide 24 | Data Center Automation with YADT | Schlomo Schapiro
YADT – Large Scale System Management
LB: www.immobilienscout24.de
Web Server
HTTPD
Tomcatweb-app
httpd
config
tomcat
App Server
Java App
app
config
Web Server
HTTPD
Tomcatweb-app
httpd
config
tomcat
App Server
Java App
app
config
Slide 25 | Data Center Automation with YADT | Schlomo Schapiro
YADT – Large Scale System Management
LB: www.immobilienscout24.de
Web Server
HTTPD
Tomcatweb-app
httpd
config
tomcat
App Server
Java App
app
config
Web Server
HTTPD
Tomcatweb-app
httpd
config
tomcat
App Server
Java App
app
config
1st
chun
k
Slide 26 | Data Center Automation with YADT | Schlomo Schapiro
YADT – Large Scale System Management
LB: www.immobilienscout24.de
Web Server
HTTPD
Tomcatweb-app
httpd
config
tomcat
App Server
Java App
app
config
Web Server
HTTPD
Tomcatweb-app
httpd
config
tomcat
App Server
Java App
app
config
1st
chun
k
2nd chunk
Slide 27 | Data Center Automation with YADT | Schlomo Schapiro
Systems Management with YADT
Reduce Systemic Complexity
Keep It Simple
Slide 28 | Data Center Automation with YADT | Schlomo Schapiro
Automation is our Continuous Delivery Platform
BusinessDecisionTo GoLive
All Humans are on the Same Side
BuildConfigDeployTestSystems-Management
Automation
Scale OutData Centers
Slide 29 | Data Center Automation with YADT | Schlomo Schapiro
DevOps
We build it togetheras one team.
We understandit completely.
And it is fully automated.
Slide 30 | Data Center Automation with YADT | Schlomo Schapiro
www.yadt-project.org
Stand 7.1C145
Slide 31 | Data Center Automation with YADT | Schlomo Schapirogo.schapiro.org/slides
Slide 32 | Data Center Automation with YADT | Schlomo Schapiro
Kontakt:Immobilien Scout GmbHAndreasstraße 1010243 Berlin
Fon: +49 30 243 01-1229 Email: [email protected]: www.immobilienscout24.de
Thank you very much!Please contact me for further questions and discussions.
All images are either public domain, used in appropriate context or taken from openclipart.org