![Page 1: Managing AWS infrastructure using CloudFormation](https://reader035.vdocument.in/reader035/viewer/2022081503/588a5ef11a28ab7a768b63a3/html5/thumbnails/1.jpg)
AWS User Group Norway
http://meetup.com/AWS-User-Group-Norway/
Managing AWS infrastructure using CloudFormation by Anton Babenko
23.11.2015Oslo, Norway
![Page 2: Managing AWS infrastructure using CloudFormation](https://reader035.vdocument.in/reader035/viewer/2022081503/588a5ef11a28ab7a768b63a3/html5/thumbnails/2.jpg)
About the group
Anders BjørnestadAnton Babenko
Arne Solheim Olaf Skaug
“This is a group for people interested in Amazon Web Services. Anyone can participate, ranging from AWS evangelists to the curious. The main focus of the group is to build up a community around AWS with socializing and talks on topics like innovations, development and possibilities. Having trouble with a stack? Maybe you'll meet someone with a solution or an approach that you haven't already tried.”
AWS Certified SysOps,Sr. Software Eng at Telenor Digital linkedin.com/in/antonbabenko [email protected]
AWS Certified Architect and Developer, Sr. Consultant at Webstep linkedin.com/in/abjoerne [email protected]
Organizers:
Cloud Architect at Nordcloud Ltd
linkedin.com/in/olafskaug [email protected]
CTO at Nordeca Insight
linkedin.com/in/arnesolheim [email protected]
![Page 3: Managing AWS infrastructure using CloudFormation](https://reader035.vdocument.in/reader035/viewer/2022081503/588a5ef11a28ab7a768b63a3/html5/thumbnails/3.jpg)
Today’s meeting1) News from AWS
2) Anton Babenko - Managing AWS infrastructure using CloudFormation
3) Arne Solheim - CloudfrontPizza and drinks sponsored by:
23NOV2015
![Page 4: Managing AWS infrastructure using CloudFormation](https://reader035.vdocument.in/reader035/viewer/2022081503/588a5ef11a28ab7a768b63a3/html5/thumbnails/4.jpg)
![Page 5: Managing AWS infrastructure using CloudFormation](https://reader035.vdocument.in/reader035/viewer/2022081503/588a5ef11a28ab7a768b63a3/html5/thumbnails/5.jpg)
Agenda:Basics of CloudFormation
Use cases
Best practices
Challenges and limitations
![Page 6: Managing AWS infrastructure using CloudFormation](https://reader035.vdocument.in/reader035/viewer/2022081503/588a5ef11a28ab7a768b63a3/html5/thumbnails/6.jpg)
AWS CloudFormation
“... an easy way to create and manage a collection of related AWS resources, provisioning and updating them in an orderly and predictable fashion.”
https://aws.amazon.com/cloudformation/
![Page 7: Managing AWS infrastructure using CloudFormation](https://reader035.vdocument.in/reader035/viewer/2022081503/588a5ef11a28ab7a768b63a3/html5/thumbnails/7.jpg)
![Page 8: Managing AWS infrastructure using CloudFormation](https://reader035.vdocument.in/reader035/viewer/2022081503/588a5ef11a28ab7a768b63a3/html5/thumbnails/8.jpg)
{ "AWSTemplateFormatVersion" : "2010-09-09", "Description" : "JSON string", "Metadata" : { template metadata }, "Parameters" : { set of parameters }, "Mappings" : { set of mappings }, "Conditions" : { set of conditions }, "Resources" : { set of resources - REQUIRED }, "Outputs" : { set of outputs }}
Template anatomy
![Page 9: Managing AWS infrastructure using CloudFormation](https://reader035.vdocument.in/reader035/viewer/2022081503/588a5ef11a28ab7a768b63a3/html5/thumbnails/9.jpg)
● Collection of AWS resources
● Supports:
○ Create
○ Update
○ Describe
○ Delete
Stack
![Page 10: Managing AWS infrastructure using CloudFormation](https://reader035.vdocument.in/reader035/viewer/2022081503/588a5ef11a28ab7a768b63a3/html5/thumbnails/10.jpg)
Manage resources
Bootstrapping of instances
Application deployment
Use cases
![Page 11: Managing AWS infrastructure using CloudFormation](https://reader035.vdocument.in/reader035/viewer/2022081503/588a5ef11a28ab7a768b63a3/html5/thumbnails/11.jpg)
Manage AWS resources
Initial bootstrapping of instances
Use Ansible, Puppet, Chef or AWS OpsWorks for more real things
Application deployment
Consider using AWS alternatives also (CodeDeploy, Elastic Beanstalk, ECS)
Use cases
![Page 12: Managing AWS infrastructure using CloudFormation](https://reader035.vdocument.in/reader035/viewer/2022081503/588a5ef11a28ab7a768b63a3/html5/thumbnails/12.jpg)
Use generator tool (JSON is not for humans)
Python: https://github.com/cloudtools/troposphere
Ruby: https://github.com/sparkleformation/sparkle_formation
Ruby: https://github.com/tongueroo/lono
Scala: https://github.com/MonsantoCo/cloudformation-template-generator
Manage AWS resources
![Page 13: Managing AWS infrastructure using CloudFormation](https://reader035.vdocument.in/reader035/viewer/2022081503/588a5ef11a28ab7a768b63a3/html5/thumbnails/13.jpg)
Use generator tools - user-data escaping hell
Ruby example 2:
:UserData => base64(interpolate(file('userdata.sh')))
Ruby example 1 (erb template):
"UserData": { "Fn::Base64": { "Fn::Join": [ "", [ <%= user_data('userdata.sh.erb') %> ] ] }
Cloudformation JSON example 1:
"UserData": { "Fn::Base64": { "Fn::Join": [ "", ["#!/bin/bash -ex","\n", "yum update -y","\n", "# here is my long shell script. Imagine how long it can be :)"] ] }}
Cloudformation JSON example 2:
"UserData": "here is long & base64 encoded string. Imagine how long it can be :)"
![Page 14: Managing AWS infrastructure using CloudFormation](https://reader035.vdocument.in/reader035/viewer/2022081503/588a5ef11a28ab7a768b63a3/html5/thumbnails/14.jpg)
Use Cloudformer to create templates based on existing AWS resources
CloudFormer: https://aws.amazon.com/developertools/6460180344805680
I have created AWS resources manually
![Page 15: Managing AWS infrastructure using CloudFormation](https://reader035.vdocument.in/reader035/viewer/2022081503/588a5ef11a28ab7a768b63a3/html5/thumbnails/15.jpg)
![Page 16: Managing AWS infrastructure using CloudFormation](https://reader035.vdocument.in/reader035/viewer/2022081503/588a5ef11a28ab7a768b63a3/html5/thumbnails/16.jpg)
![Page 17: Managing AWS infrastructure using CloudFormation](https://reader035.vdocument.in/reader035/viewer/2022081503/588a5ef11a28ab7a768b63a3/html5/thumbnails/17.jpg)
![Page 18: Managing AWS infrastructure using CloudFormation](https://reader035.vdocument.in/reader035/viewer/2022081503/588a5ef11a28ab7a768b63a3/html5/thumbnails/18.jpg)
CloudFormation Designer view of CloudFormer stack
Template: https://s3.amazonaws.com/cloudformation-templates-eu-west-1/CloudFormer.template
![Page 19: Managing AWS infrastructure using CloudFormation](https://reader035.vdocument.in/reader035/viewer/2022081503/588a5ef11a28ab7a768b63a3/html5/thumbnails/19.jpg)
3 ways to organize stacks
![Page 20: Managing AWS infrastructure using CloudFormation](https://reader035.vdocument.in/reader035/viewer/2022081503/588a5ef11a28ab7a768b63a3/html5/thumbnails/20.jpg)
Fit small deployments
Small size limit (50kb when load from local file; 450kb when load from S3)
1.Stand-alone stacks
![Page 21: Managing AWS infrastructure using CloudFormation](https://reader035.vdocument.in/reader035/viewer/2022081503/588a5ef11a28ab7a768b63a3/html5/thumbnails/21.jpg)
Easy execution - aws-cli
Allow to put reusable parts into separate stacks
One fails = all fails = all rollback
Pass parameters to the internal stacks through parent
Reference to resources in parent: { "Fn::GetAtt" : [ "myVPCStack", "Outputs.VPCId" ] }
2.Nested stacks
![Page 22: Managing AWS infrastructure using CloudFormation](https://reader035.vdocument.in/reader035/viewer/2022081503/588a5ef11a28ab7a768b63a3/html5/thumbnails/22.jpg)
Manual execution one after another
Pass outputs as inputs to downstream template
Pass parameters to each stack directly
Independent failure = independent rollback
Can’t use WaitContitions with other stacks
3.Pipelined stacks
![Page 23: Managing AWS infrastructure using CloudFormation](https://reader035.vdocument.in/reader035/viewer/2022081503/588a5ef11a28ab7a768b63a3/html5/thumbnails/23.jpg)
Continuous Integration: Infrastructure & Application
Infrastructure:● Parametrize everything● Validate templates (AWS CLI)● Version templates● Probably run on separate AWS account first (CI)
Application deployment:● EC2 cloud-init + new application version = new deploy
![Page 24: Managing AWS infrastructure using CloudFormation](https://reader035.vdocument.in/reader035/viewer/2022081503/588a5ef11a28ab7a768b63a3/html5/thumbnails/24.jpg)
CloudFormation challenges
● Can’t import already created resources without deleting them first● Not all AWS resources/features/services are supported by CloudFormation (eg, EC2 keypairs)● No officially supported CloudFormation generator available● No way to see what kind of changes are going to be applied● Failed state… what to do ?
○ Do not update resources created by CloudFormation manually
![Page 25: Managing AWS infrastructure using CloudFormation](https://reader035.vdocument.in/reader035/viewer/2022081503/588a5ef11a28ab7a768b63a3/html5/thumbnails/25.jpg)
CloudFormation limitation
● JSON format is not very human-friendly● No iteration and limited conditional support (and/or/not/equal)● Limited ability to adjust stack based on dynamic conditions ● Managing dependencies between templates
○ Tieing together inputs/outputs is not directly supported
![Page 26: Managing AWS infrastructure using CloudFormation](https://reader035.vdocument.in/reader035/viewer/2022081503/588a5ef11a28ab7a768b63a3/html5/thumbnails/26.jpg)
Summary
● Use JSON generators● Keep templates maintainable and single purposed● Probably start with stand-alone stack and iterate● Decide how you can handle failures (rollback just one stack or all)● Integrate with CI similar way how you do with your application● Use CloudFormation for very primitive application deployments
![Page 27: Managing AWS infrastructure using CloudFormation](https://reader035.vdocument.in/reader035/viewer/2022081503/588a5ef11a28ab7a768b63a3/html5/thumbnails/27.jpg)
Available templates: https://aws.amazon.com/cloudformation/aws-cloudformation-templates/
CloudFormation Masterclass: http://www.slideshare.net/AmazonWebServices/aws-cloudformation-masterclass
(DVO304) AWS CloudFormation Best Practices: http://www.slideshare.net/AmazonWebServices/dvo304-aws-cloudformation-best-practices
Links
![Page 28: Managing AWS infrastructure using CloudFormation](https://reader035.vdocument.in/reader035/viewer/2022081503/588a5ef11a28ab7a768b63a3/html5/thumbnails/28.jpg)
Questions ?
![Page 29: Managing AWS infrastructure using CloudFormation](https://reader035.vdocument.in/reader035/viewer/2022081503/588a5ef11a28ab7a768b63a3/html5/thumbnails/29.jpg)
Thank you!
See you at DevOps Norway meetup 14th of December 2015:
Manage AWS infrastructure (as code) using Terraform
http://www.meetup.com/DevOps-Norway/events/226820193/