puppet camp sydney 2014 - evolving design patterns in aws
DESCRIPTION
A view of the past, present and future roles and architectures of Puppet Enterprise in AWS. Based on real world enterprise examples this presentation gives a in-the-trenches view of 4 key architectural patterns for Puppet Enterprise in AWS. Architecture and demo of native integration between Puppet Enterprise and AWS Autoscaling. Dynamic autoscaled nodes are automatically signed (programatically, NOT via auto sign), and groups assigned according to AWS native metadata. Nodes are then deregistered and remove from the master as autoscale/autoheal deregisters them. Solution 1 - Multi Master Solution 2 - Puppet Controlling AWS - Uplift of Puppet CloudPack to support AWS, Rackspace and Joyent. Solution 3 - Masterless puppet via AWS S3 and local apply. Solution 4 - Puppet Enterprise natively interfacing with AWS Autoscale via the Sourced Autoscale Broker. Solution 5 - A Scale out architecture for autoscaled PaaS platforms with Puppet Enterprise providing a compliance tier.TRANSCRIPT
![Page 1: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/1.jpg)
E V O LV I N G D E S I G N PAT T E R N S I N A W S
![Page 2: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/2.jpg)
John Painter James DymondSolutions Architect Infrastructure Engineer
sourcedgroup.com.au
www.linkedin.com/in/painterj www.linkedin.com/in/jamesdymond
![Page 3: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/3.jpg)
Sourced Group architect and deploy a globally distributed multi-master puppet solution for a large investment bank
February 2011
![Page 4: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/4.jpg)
O P E R AT I N G S Y S T E M S
A P P L I C AT I O N C O N F I G U R AT I O N
Converge the infrastructure to one point of truth
![Page 5: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/5.jpg)
A U D I T I N G G O V E R N A N C E S I M P L I F I E D C O N T R O L
Control and report on the point of truth
![Page 6: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/6.jpg)
Scale that convergence to the entire enterprise
A U D I T I N G
G O V E R N A N C E
S I M P L I F I E D C O N T R O L
![Page 7: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/7.jpg)
E U R O P E : 6 P M - 8 A MU S A : 6 P M - 8 A MA S I A : 6 P M - 8 A M
Global risk management, simplified releases
![Page 8: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/8.jpg)
Sourced Group architect and deploy a globally distributed multi-master puppet solution for a large investment bank
February 2011
![Page 9: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/9.jpg)
“Building a Multimaster Environment”
Greg Cockburn
PuppetCamp Sydney 2012
![Page 10: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/10.jpg)
http://www.slideshare.net/gergnz/puppetcamp-sydney-2012-building-a-multimaster-environment
http://www.slideshare.net/gergnz/puppetcamp-sydney-2012-building-a-multimaster-environment
![Page 11: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/11.jpg)
“Building a Multimaster Environment”
Greg Cockburn
PuppetCamp Sydney 2012
![Page 12: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/12.jpg)
Sourced are the first to onboard an AsiaPac telecommunication company onto AWS, powered by Puppet Enterprise
February 2012
![Page 13: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/13.jpg)
E V E R Y T H I N G I S P R O G R A M AT I C
S TA N D A R D I S AT I O N O F I N F R A S T R U C T U R E
![Page 14: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/14.jpg)
Puppet CloudPack
![Page 15: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/15.jpg)
V I R T U A L P R I VAT E C L O U D P R I VAT E I P S PA C E
C O M P U T E C O M P U T E
C O M P U T E C O M P U T E
C U S T O M E R
V P N
C O M P U T E C O M P U T E
C O M P U T E C O M P U T E
Public IP Public IP
Public IP Public IP
Public IP
![Page 16: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/16.jpg)
Sourced are the first to onboard an AsiaPac telecommunication company onto AWS, powered by Puppet Enterprise
February 2012
![Page 17: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/17.jpg)
“Using Puppet Enterprise Edition as Heterogeneous
Cloud Glue”
PuppetConf 2012
![Page 18: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/18.jpg)
C L O U D PA C K U P L I F T
• Uplifted the existing Puppet CloudPack to support EC2 instances inside the AWS VPC
• Left Puppet Enterprise in the middle of the provisioning and lifecycle management
• Supported Multi-cloud provisioning
• http://www.youtube.com/watch?v=mwiBjJZWraA
![Page 19: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/19.jpg)
S TA N D A R D I S AT I O N A N D C O N S O L I D AT I O N
![Page 20: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/20.jpg)
“Using Puppet Enterprise Edition as Heterogeneous
Cloud Glue”
PuppetConf 2012
![Page 21: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/21.jpg)
Sourced implement a masterless Enterprise Puppet solution for a financial services organisation
March 2013
![Page 22: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/22.jpg)
C L O U D C O M P U T I N G I S F U N D A M E N TA L LY D I F F E R E N T
![Page 23: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/23.jpg)
T R A N S I E N T C O M P U T E
![Page 24: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/24.jpg)
V 1 . 0 V 1 . 0
myapp.com.au
![Page 25: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/25.jpg)
V 1 . 0 V 1 . 0
myapp.com.au
V 2 . 0 V 2 . 0
![Page 26: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/26.jpg)
myapp.com.au
V 2 . 0 V 2 . 0V 1 . 0 V 1 . 0
![Page 27: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/27.jpg)
Infrastructure, deployment, and configuration have standardised
!
It was not just servers that were transient, it was entire application stacks
![Page 28: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/28.jpg)
Enterprises want to leverage this agility to drive their application iteration harder and respond to
market conditions.
Cloud computing has significantly changed the application lifecycle
![Page 29: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/29.jpg)
Weekly/Monthly -> Hourly
![Page 30: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/30.jpg)
+
C L O U D F O R M AT I O N
Tasks for AWSV 2 . 0 V 2 . 0
![Page 31: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/31.jpg)
• Cloudformation (AWS)
• Cloud Init (Puppet)
• CFN Init (AWS)
• Completely isolated application delivery
• Aware of the other AWS elements around them
![Page 32: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/32.jpg)
– N I G E L K E R S T E N - P U P P E T C A M P S Y D N E Y 2 0 1 4
“18 months really sucks”
![Page 33: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/33.jpg)
- E V E R Y S I N G L E P R O D U C T M A N A G E R . E V E R .
“18 months really sucks”
![Page 34: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/34.jpg)
Business teams are willing to accept standardisation to gain agility and control over deployments
![Page 35: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/35.jpg)
A P P L I C AT I O N
A P P L I C AT I O N
A P P L I C AT I O N
A P P A P P A P P
A P P A P P A P P
A P P A P P A P P
Standard Framework Lots of AWS Features
![Page 36: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/36.jpg)
Application Configuration
Configuration Managent
Governance and Compliance
![Page 37: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/37.jpg)
Configuration Managent
Governance and ComplianceApplication Configuration
![Page 38: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/38.jpg)
Load
Time
![Page 39: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/39.jpg)
Load
Time
![Page 40: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/40.jpg)
Load
Time
$
$
$ $
$
$
![Page 41: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/41.jpg)
G I T
I N S TA N C E I N S TA N C E
C D T O O L
AW S S 3
L O G G I N G
I N S TA N C EI N S TA N C E
![Page 42: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/42.jpg)
S O L U T I O N 4
![Page 43: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/43.jpg)
– N I G E L K E R S T E N - P U P P E T C A M P S Y D N E Y 2 0 1 4
“If you not already using PuppetDB, you should be.”
![Page 44: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/44.jpg)
H Y B R I D C L O U D
![Page 45: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/45.jpg)
H Y B R I D C L O U D
![Page 46: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/46.jpg)
P U P P E T A W S A U T O S C A L I N G B R O K E R
![Page 47: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/47.jpg)
I N S TA N C E I N S TA N C E I N S TA N C E
Simple Notification Service (SNS)
Autoscaling Group
SNS Topic
Simple Queue Service (SQS)
…
A U T O S C A L E B R O K E R
![Page 48: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/48.jpg)
D E M O
![Page 49: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/49.jpg)
S O L U T I O N 5
![Page 50: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/50.jpg)
O P E R AT I N G S Y S T E M S
O P E R AT I N G S Y S T E M S
O P E R AT I N G S Y S T E M S
A P P L I C AT I O N
A P P L I C AT I O N
A P P L I C AT I O N
Framework
![Page 51: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/51.jpg)
Rather than making the OS a good candidate for the application, we are now tailoring the applications to be a good citizen of a standardised compute platform.
![Page 52: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/52.jpg)
H A R D WA R E
O P E R AT I N G S Y S T E M
R U N T I M E
A P P L I C AT I O N
![Page 53: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/53.jpg)
H A R D WA R E
O P E R AT I N G S Y S T E M
R U N T I M E
A P P L I C AT I O N
![Page 54: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/54.jpg)
O P E R AT I N G S Y S T E M S
O P E R AT I N G S Y S T E M S
O P E R AT I N G S Y S T E M S
A P P L I C AT I O N
A P P L I C AT I O N
A P P L I C AT I O N
Framework
![Page 55: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/55.jpg)
- T H E N E W P R O D U C T M A N A G E R S
“1 week really sucks”
![Page 56: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/56.jpg)
O P E N S H I F T
• Standardised runtime container
• Solution that accepts application artefacts and deploys, runs, scales, and manages their lifecycle
• Drives application portability
• OS and application containers are a blackbox
![Page 57: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/57.jpg)
T H E F R A M E W O R K S S T I L L N E E D T O G O V E R N E D
![Page 58: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/58.jpg)
A P P A P P A P P A P P
Puppet + Autoscale Broker
A M A Z O N W E B S E R V I C E S - A U T O S C A L E D
![Page 59: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/59.jpg)
A P P A P P A P P A P P
Puppet + Autoscale Broker + Global Distribution
AW S - S Y D N E Y AW S - S I N G A P O R E
![Page 60: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/60.jpg)
A P P A P P A P P A P P
Puppet + Autoscale Broker + Global Distribution + Multi Cloud
AW S - S Y D N E Y AW S - S I N G A P O R E< C L O U D > - S Y D N E Y < C L O U D > - S I N G A P O R E
![Page 61: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/61.jpg)
A P P A P P A P P A P P
Puppet + Autoscale Broker + Global Distribution + Multi EVERYTHING!
AW S - S Y D N E Y AW S - S I N G A P O R E< C L O U D > - S Y D N E Y
![Page 62: Puppet Camp Sydney 2014 - Evolving Design Patterns in AWS](https://reader034.vdocument.in/reader034/viewer/2022052622/559428ba1a28abf5408b467f/html5/thumbnails/62.jpg)
John Painter James DymondSolutions Architect Infrastructure Engineer
sourcedgroup.com.au
www.linkedin.com/in/painterj www.linkedin.com/in/jamesdymond