driving devops transformation in enterprises · 2021. 3. 11. · applying devops make engineering...
TRANSCRIPT
![Page 1: Driving DevOps Transformation in Enterprises · 2021. 3. 11. · Applying DevOps Make engineering running for a shared goal Change the system frequently in small iterations Automate](https://reader035.vdocument.in/reader035/viewer/2022062510/6134721cdfd10f4dd73bbc51/html5/thumbnails/1.jpg)
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Mark Rambow
Software Development Manager, AWS OpsWorks, Berlin
Driving DevOps Transformation
in Enterprises
![Page 2: Driving DevOps Transformation in Enterprises · 2021. 3. 11. · Applying DevOps Make engineering running for a shared goal Change the system frequently in small iterations Automate](https://reader035.vdocument.in/reader035/viewer/2022062510/6134721cdfd10f4dd73bbc51/html5/thumbnails/2.jpg)
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
acts_as_enterprisey
![Page 3: Driving DevOps Transformation in Enterprises · 2021. 3. 11. · Applying DevOps Make engineering running for a shared goal Change the system frequently in small iterations Automate](https://reader035.vdocument.in/reader035/viewer/2022062510/6134721cdfd10f4dd73bbc51/html5/thumbnails/3.jpg)
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
start up
![Page 4: Driving DevOps Transformation in Enterprises · 2021. 3. 11. · Applying DevOps Make engineering running for a shared goal Change the system frequently in small iterations Automate](https://reader035.vdocument.in/reader035/viewer/2022062510/6134721cdfd10f4dd73bbc51/html5/thumbnails/4.jpg)
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
enterprises
![Page 5: Driving DevOps Transformation in Enterprises · 2021. 3. 11. · Applying DevOps Make engineering running for a shared goal Change the system frequently in small iterations Automate](https://reader035.vdocument.in/reader035/viewer/2022062510/6134721cdfd10f4dd73bbc51/html5/thumbnails/5.jpg)
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
enterprises and monolith software
![Page 6: Driving DevOps Transformation in Enterprises · 2021. 3. 11. · Applying DevOps Make engineering running for a shared goal Change the system frequently in small iterations Automate](https://reader035.vdocument.in/reader035/viewer/2022062510/6134721cdfd10f4dd73bbc51/html5/thumbnails/6.jpg)
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
DevOps
![Page 7: Driving DevOps Transformation in Enterprises · 2021. 3. 11. · Applying DevOps Make engineering running for a shared goal Change the system frequently in small iterations Automate](https://reader035.vdocument.in/reader035/viewer/2022062510/6134721cdfd10f4dd73bbc51/html5/thumbnails/7.jpg)
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Drive securely and fast
![Page 8: Driving DevOps Transformation in Enterprises · 2021. 3. 11. · Applying DevOps Make engineering running for a shared goal Change the system frequently in small iterations Automate](https://reader035.vdocument.in/reader035/viewer/2022062510/6134721cdfd10f4dd73bbc51/html5/thumbnails/8.jpg)
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
A culture clash
Developers goal: release customer features
Operators goal: keep the lights on
Never change a running system?
![Page 9: Driving DevOps Transformation in Enterprises · 2021. 3. 11. · Applying DevOps Make engineering running for a shared goal Change the system frequently in small iterations Automate](https://reader035.vdocument.in/reader035/viewer/2022062510/6134721cdfd10f4dd73bbc51/html5/thumbnails/9.jpg)
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Development and Operations teams
(should) own a business goal
![Page 10: Driving DevOps Transformation in Enterprises · 2021. 3. 11. · Applying DevOps Make engineering running for a shared goal Change the system frequently in small iterations Automate](https://reader035.vdocument.in/reader035/viewer/2022062510/6134721cdfd10f4dd73bbc51/html5/thumbnails/10.jpg)
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
DevOps requires
organizational changes
![Page 11: Driving DevOps Transformation in Enterprises · 2021. 3. 11. · Applying DevOps Make engineering running for a shared goal Change the system frequently in small iterations Automate](https://reader035.vdocument.in/reader035/viewer/2022062510/6134721cdfd10f4dd73bbc51/html5/thumbnails/11.jpg)
Agenda
How organizational structures reflect in infrastructure
How to separate Orgs and Teams
Above and Beyond, a server per team
Cross organizational policies and cookbooks
Demo: How to organize cookbook development
Two Pizza Teams at Amazon
Decentralization
Autonomy
Accountability
Ownership
DevOps
![Page 12: Driving DevOps Transformation in Enterprises · 2021. 3. 11. · Applying DevOps Make engineering running for a shared goal Change the system frequently in small iterations Automate](https://reader035.vdocument.in/reader035/viewer/2022062510/6134721cdfd10f4dd73bbc51/html5/thumbnails/12.jpg)
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
DevOps pillars
Informational silos
Accept failures
Frequent and small changes
Automation, tooling
Monitoring
![Page 13: Driving DevOps Transformation in Enterprises · 2021. 3. 11. · Applying DevOps Make engineering running for a shared goal Change the system frequently in small iterations Automate](https://reader035.vdocument.in/reader035/viewer/2022062510/6134721cdfd10f4dd73bbc51/html5/thumbnails/13.jpg)
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
DevOps pillars
Informational silos
Accept failures
Frequent and small changes
Automation, tooling
Monitoring
![Page 14: Driving DevOps Transformation in Enterprises · 2021. 3. 11. · Applying DevOps Make engineering running for a shared goal Change the system frequently in small iterations Automate](https://reader035.vdocument.in/reader035/viewer/2022062510/6134721cdfd10f4dd73bbc51/html5/thumbnails/14.jpg)
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
DevOps: Automation and Tooling
![Page 15: Driving DevOps Transformation in Enterprises · 2021. 3. 11. · Applying DevOps Make engineering running for a shared goal Change the system frequently in small iterations Automate](https://reader035.vdocument.in/reader035/viewer/2022062510/6134721cdfd10f4dd73bbc51/html5/thumbnails/15.jpg)
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Automation and Tooling in the cloud
Public clouds make infrastructure behave like software
No need for complicated capacity planning
Services make applications scale
Failover and Scaling in minutes
Infrastructure as code
![Page 16: Driving DevOps Transformation in Enterprises · 2021. 3. 11. · Applying DevOps Make engineering running for a shared goal Change the system frequently in small iterations Automate](https://reader035.vdocument.in/reader035/viewer/2022062510/6134721cdfd10f4dd73bbc51/html5/thumbnails/16.jpg)
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS CloudFormation
• Infrastructure as code for AWS Services
• Similar to Terraform but AWS optimized
• Templates in YAML or JSON
Amazon
VPC
AWS
Lambda
Amazon
EC2
Amazon
S3
Amazon
DynamoDBElastic Load
Balancing
Amazon
CloudFront
Amazon
CloudWatch
Elastic Load
Balancing
AWS
Elastic
Beanstalk
Amazon
ECSAmazon
RDSAmazon
RedshiftAmazon
VPC
![Page 17: Driving DevOps Transformation in Enterprises · 2021. 3. 11. · Applying DevOps Make engineering running for a shared goal Change the system frequently in small iterations Automate](https://reader035.vdocument.in/reader035/viewer/2022062510/6134721cdfd10f4dd73bbc51/html5/thumbnails/17.jpg)
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Infrastructure as Code workflow
Code Version Control Build / Test Deploy
Vim
Notepad
Eclipse
AWS Cloud9
Git
SVN
MS Github
AWS CodeCommit
Linting
Testing
Staging
Load testing
Infrastructure
Hosts, Services, …
![Page 18: Driving DevOps Transformation in Enterprises · 2021. 3. 11. · Applying DevOps Make engineering running for a shared goal Change the system frequently in small iterations Automate](https://reader035.vdocument.in/reader035/viewer/2022062510/6134721cdfd10f4dd73bbc51/html5/thumbnails/18.jpg)
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Infrastructure as Code workflow
Code Version Control Build / Test Deploy
Vim
Notepad
Eclipse
AWS Cloud9
Git
SVN
MS Github
AWS CodeCommit
Linting
Testing
Staging
Load testing
Infrastructure
Hosts, Services, …
CI/CD System
![Page 19: Driving DevOps Transformation in Enterprises · 2021. 3. 11. · Applying DevOps Make engineering running for a shared goal Change the system frequently in small iterations Automate](https://reader035.vdocument.in/reader035/viewer/2022062510/6134721cdfd10f4dd73bbc51/html5/thumbnails/19.jpg)
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
CloudFormation concepts and technology
Framework
Stack creation
Stack updates
Error detection and rollback
Configured AWS resources
Service event aware
Customizable
Template CloudFormation Stack
JSON/YAML formatted file
Parameter definition
Resource creation
Configuration actions
![Page 20: Driving DevOps Transformation in Enterprises · 2021. 3. 11. · Applying DevOps Make engineering running for a shared goal Change the system frequently in small iterations Automate](https://reader035.vdocument.in/reader035/viewer/2022062510/6134721cdfd10f4dd73bbc51/html5/thumbnails/20.jpg)
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Bootstrapping Servers
"Resources" : {
"Ec2Instance" : {
"Type" : "AWS::EC2::Instance",
"Properties" : {
"KeyName" : { "Ref" : "KeyName" },
"SecurityGroups" : [ { "Ref" : "InstanceSecurityGroup" } ],
"ImageId" : "ami-79fd7eee"
}
}
![Page 21: Driving DevOps Transformation in Enterprises · 2021. 3. 11. · Applying DevOps Make engineering running for a shared goal Change the system frequently in small iterations Automate](https://reader035.vdocument.in/reader035/viewer/2022062510/6134721cdfd10f4dd73bbc51/html5/thumbnails/21.jpg)
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Bootstrapping Servers with Userdata
"Resources" : {
"Ec2Instance" : {
"Type" : "AWS::EC2::Instance",
"Properties" : {
"KeyName" : { "Ref" : "KeyName" },
"SecurityGroups" : [ { "Ref" : "InstanceSecurityGroup" } ],
"ImageId" : { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "AMI" ]},
"UserData" : { "Fn::Base64" : { "Fn::Join" : ["",[
"#!/bin/bash -ex","\n",
"curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.33.8/install.sh | bash","
"curl --silent --location https://rpm.nodesource.com/setup_6.x | bash -”,”\n”
"yum -y install nodejs",”\n”
"npm i -g [email protected]",”\n”
"curl 'https://gist.githubusercontent.com/example/5c035301e13037e52cd689205b08c121/raw/e22a4606401ce63af715792b3fe50ef869b0557f/hello
"pm2 start /home/ec2-user/hello-world-server.js -i 0 --name 'node-app'","\n"
]]}}
}
}
![Page 22: Driving DevOps Transformation in Enterprises · 2021. 3. 11. · Applying DevOps Make engineering running for a shared goal Change the system frequently in small iterations Automate](https://reader035.vdocument.in/reader035/viewer/2022062510/6134721cdfd10f4dd73bbc51/html5/thumbnails/22.jpg)
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Visualizing with CloudFormation templates
• AWS CloudFormation Designer
• Visualize template resources
• Modify template with drag-drop
• Customize sample templates
![Page 23: Driving DevOps Transformation in Enterprises · 2021. 3. 11. · Applying DevOps Make engineering running for a shared goal Change the system frequently in small iterations Automate](https://reader035.vdocument.in/reader035/viewer/2022062510/6134721cdfd10f4dd73bbc51/html5/thumbnails/23.jpg)
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
CloudFormation
Templated infrastructure
Versioned
Tested (Linting + TaskCat)
Repeatable + Saleable
Managed decencies
Updates your environment
![Page 24: Driving DevOps Transformation in Enterprises · 2021. 3. 11. · Applying DevOps Make engineering running for a shared goal Change the system frequently in small iterations Automate](https://reader035.vdocument.in/reader035/viewer/2022062510/6134721cdfd10f4dd73bbc51/html5/thumbnails/24.jpg)
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Configuration Management
Manages lifecycle after EC2 provisioning
Transforms a server into an App, Web, Database, ... Server
Abstraction
Portability
Patches, updates
New software
Configurations updates
Managing access
![Page 25: Driving DevOps Transformation in Enterprises · 2021. 3. 11. · Applying DevOps Make engineering running for a shared goal Change the system frequently in small iterations Automate](https://reader035.vdocument.in/reader035/viewer/2022062510/6134721cdfd10f4dd73bbc51/html5/thumbnails/25.jpg)
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Configuration Management
Managing changes on hosts
Run commands across all hosts
Monitoring and Insights for all running resources
Enforcing Compliance and Policies
Every single bit on the instance
File
s
Package
s
Configuratio
n
Key
s
![Page 26: Driving DevOps Transformation in Enterprises · 2021. 3. 11. · Applying DevOps Make engineering running for a shared goal Change the system frequently in small iterations Automate](https://reader035.vdocument.in/reader035/viewer/2022062510/6134721cdfd10f4dd73bbc51/html5/thumbnails/26.jpg)
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Configuration Management
Puppet, Chef
DSL for host configuration
Abstraction for resources (packages, files, secrets)
Scriptable, it’s Ruby code
Community with configuration for everything
![Page 27: Driving DevOps Transformation in Enterprises · 2021. 3. 11. · Applying DevOps Make engineering running for a shared goal Change the system frequently in small iterations Automate](https://reader035.vdocument.in/reader035/viewer/2022062510/6134721cdfd10f4dd73bbc51/html5/thumbnails/27.jpg)
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
DSL (installing NGinX)
include_recipe "apt"
package "nginx" do
action :install
end
service "nginx" do
action [ :enable, :start ]
end
cookbook_file "/www/index.html" do
source "index.html"
mode "0644"
end
Chefclass nginx {
package { "nginx":
ensure => installed,
}
service { "nginx":
ensure => running,
require => Package["nginx"],
}
file { "/www/index.html":
source =>
"puppet:///modules/n/index.html",
notify => Service["nginx"],
}
}
Puppet
![Page 28: Driving DevOps Transformation in Enterprises · 2021. 3. 11. · Applying DevOps Make engineering running for a shared goal Change the system frequently in small iterations Automate](https://reader035.vdocument.in/reader035/viewer/2022062510/6134721cdfd10f4dd73bbc51/html5/thumbnails/28.jpg)
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS OpsWorks
OpsWorks Stacks
Manage any application with Chef
OpsWorks for Chef Automate and Puppet Enterprise
Managed Chef Automate or Puppet Enterprise
Server in your Account, full root access
Undifferentiated Chef/Puppet experience
Support through AWS
Pay as you go, free tier of 10 nodes
![Page 29: Driving DevOps Transformation in Enterprises · 2021. 3. 11. · Applying DevOps Make engineering running for a shared goal Change the system frequently in small iterations Automate](https://reader035.vdocument.in/reader035/viewer/2022062510/6134721cdfd10f4dd73bbc51/html5/thumbnails/29.jpg)
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
OpsWorks for Chef Automate / Puppet Enterprise
Maintenance by OpsWorks
Continuous Health Checks
Automated and manual backups
Restoration and Recreation
Weekly security patches of the server
Automatic upgrades
![Page 30: Driving DevOps Transformation in Enterprises · 2021. 3. 11. · Applying DevOps Make engineering running for a shared goal Change the system frequently in small iterations Automate](https://reader035.vdocument.in/reader035/viewer/2022062510/6134721cdfd10f4dd73bbc51/html5/thumbnails/30.jpg)
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
OpsWorks for Chef Automate / Puppet Enterprise
Integrated unattended Node provisioning
IAM for permission handling (Authorization)
Userdata for ASG, CFN, EC2, Terraform, and more
StarterKit to get started quickly
![Page 31: Driving DevOps Transformation in Enterprises · 2021. 3. 11. · Applying DevOps Make engineering running for a shared goal Change the system frequently in small iterations Automate](https://reader035.vdocument.in/reader035/viewer/2022062510/6134721cdfd10f4dd73bbc51/html5/thumbnails/31.jpg)
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS CloudFormation + Configuration
Management1. AWS CloudFormation
Setting up all AWS Services
Creating all EC2 instances
2. Configuration Management (AWS OpsWorks)
Configuring EC2 instances and on-premises
Auto configure new instances
Integrated with ASGs
Monitoring and reverting drift of instances
![Page 32: Driving DevOps Transformation in Enterprises · 2021. 3. 11. · Applying DevOps Make engineering running for a shared goal Change the system frequently in small iterations Automate](https://reader035.vdocument.in/reader035/viewer/2022062510/6134721cdfd10f4dd73bbc51/html5/thumbnails/32.jpg)
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Monitoring your state
AWS OpsWorks for Chef Automate and Puppet Enterprise
Deep insights into node converges
Detecting Drift (while re-enforcing policy)
Continuous Compliance with InSpec and Chef Automate
![Page 33: Driving DevOps Transformation in Enterprises · 2021. 3. 11. · Applying DevOps Make engineering running for a shared goal Change the system frequently in small iterations Automate](https://reader035.vdocument.in/reader035/viewer/2022062510/6134721cdfd10f4dd73bbc51/html5/thumbnails/33.jpg)
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Puppet Enterprise
![Page 34: Driving DevOps Transformation in Enterprises · 2021. 3. 11. · Applying DevOps Make engineering running for a shared goal Change the system frequently in small iterations Automate](https://reader035.vdocument.in/reader035/viewer/2022062510/6134721cdfd10f4dd73bbc51/html5/thumbnails/34.jpg)
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Chef Automate
![Page 35: Driving DevOps Transformation in Enterprises · 2021. 3. 11. · Applying DevOps Make engineering running for a shared goal Change the system frequently in small iterations Automate](https://reader035.vdocument.in/reader035/viewer/2022062510/6134721cdfd10f4dd73bbc51/html5/thumbnails/35.jpg)
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Continuous Infrastructure
Development
![Page 36: Driving DevOps Transformation in Enterprises · 2021. 3. 11. · Applying DevOps Make engineering running for a shared goal Change the system frequently in small iterations Automate](https://reader035.vdocument.in/reader035/viewer/2022062510/6134721cdfd10f4dd73bbc51/html5/thumbnails/36.jpg)
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS CodePipeline and CodeBuild
• Fully managed
continuous delivery
service
• Model and monitor your
release process
• Builds, tests, and deploys
triggered by a code
change
Step
![Page 37: Driving DevOps Transformation in Enterprises · 2021. 3. 11. · Applying DevOps Make engineering running for a shared goal Change the system frequently in small iterations Automate](https://reader035.vdocument.in/reader035/viewer/2022062510/6134721cdfd10f4dd73bbc51/html5/thumbnails/37.jpg)
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS CodePipeline and CodeBuild
• Fully managed
continuous delivery
service
• Model and monitor your
release process
• Builds, tests, and deploys
triggered by a code
change
Transition
![Page 38: Driving DevOps Transformation in Enterprises · 2021. 3. 11. · Applying DevOps Make engineering running for a shared goal Change the system frequently in small iterations Automate](https://reader035.vdocument.in/reader035/viewer/2022062510/6134721cdfd10f4dd73bbc51/html5/thumbnails/38.jpg)
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS CodePipeline and CodeBuild
• Fully managed
continuous delivery
service
• Model and monitor your
release process
• Builds, tests, and deploys
triggered by a code
change
Action
![Page 39: Driving DevOps Transformation in Enterprises · 2021. 3. 11. · Applying DevOps Make engineering running for a shared goal Change the system frequently in small iterations Automate](https://reader035.vdocument.in/reader035/viewer/2022062510/6134721cdfd10f4dd73bbc51/html5/thumbnails/39.jpg)
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Continuous Infrastructure Delivery
Desired State in code
Lint and test the infrastructure before release
Rollout gradually to production
Replicate environments and regions
![Page 40: Driving DevOps Transformation in Enterprises · 2021. 3. 11. · Applying DevOps Make engineering running for a shared goal Change the system frequently in small iterations Automate](https://reader035.vdocument.in/reader035/viewer/2022062510/6134721cdfd10f4dd73bbc51/html5/thumbnails/40.jpg)
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Applying DevOps
Make engineering running for a shared goal
Change the system frequently in small iterations
Automate everything that is effort, time-consuming, toil
Treat your infrastructure like software
Versioning, testing, continuous delivery
Failure is unavoidable
monitoring and mitigations strategies are key
Buy Lunch for your Development AND Operations team
![Page 41: Driving DevOps Transformation in Enterprises · 2021. 3. 11. · Applying DevOps Make engineering running for a shared goal Change the system frequently in small iterations Automate](https://reader035.vdocument.in/reader035/viewer/2022062510/6134721cdfd10f4dd73bbc51/html5/thumbnails/41.jpg)
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Demo: AWS CodePipeline to rollout
infrastructure changes
AWS
OpsWorksAWS
CodeCommit
AWS
CodePipeline
AWS
CodeBuild
+
![Page 42: Driving DevOps Transformation in Enterprises · 2021. 3. 11. · Applying DevOps Make engineering running for a shared goal Change the system frequently in small iterations Automate](https://reader035.vdocument.in/reader035/viewer/2022062510/6134721cdfd10f4dd73bbc51/html5/thumbnails/42.jpg)
AWS Partner DevDay
Free full-day workshop in partnership with Chef Software Inc.
Sign Up Today !
Berlin, June 12 2018http://bit.ly/aws-devday-berlin
London, June 19 2018http://bit.ly/aws-devday-london