terraform at adobe - usenix...introducon 2 systems engineer @ adobe audience manager (aam) been with...
TRANSCRIPT
Terraform at Adobe KelvinJasperson
Introduc)on
2
Systems Engineer @ Adobe Audience Manager (AAM) Been with Adobe for 18 months AAM was acquired by Adobe in 2011, and is 100% in AWS Twitter- @zxjinn
HashiCorp
3
Raise your hands
• WhoknowswhatTerraformis?• WhousesTerraform?• …inproduc=on?
4
Terraform
• Infrastructureascode• Supportsmanyproviders
• AWS• Azure• DigitalOcean• GoogleCloud• Heroku• OpenStack• VMwarevSphere/vCloudDirector• others…
5
Why Terraform?
• Funtowrite• Easytoextendwithmodules• Showstheexecu=onplan(no-op)• StatestoredinacommiTablefile
6
Basic Terraform Example
Basic Terraform Example
$ cat main.tfresource "aws_instance" "app" { ami = "ami-d1f482b1" count = 5 instance_type = "t2.micro"}$ terraform plan+ aws_instance.app.0...+ aws_instance.app.1...$ terraform applyaws_instance.app.0: Creating...Apply complete! Resources: 5 added, 0 changed, 0 destroyed.$
8
It worked! Parallel, takes ~1 min
9
Basic Terraform Destroy $ terraform destroyDo you really want to destroy? Terraform will delete all your managed infrastructure. There is no undo. Only 'yes' will be accepted to confirm. Enter a value:
10
yes
aws_instance.app.0: Destroying...Apply complete! Resources: 0 added, 0 changed, 5 destroyed.$
It worked! Parallel, takes ~1 min
11
More than just EC2 instances
• S3-SimpleStorageService• CloudForma=on• VPC-VirtualPrivateCloud• SQS-SimpleQueueService• Route53-HostedDNS• RDS-Rela=onalDatabaseService• IAM-Iden=tyandAccessManagement• ECS-EC2ContainerService• others…
12
Modules, Compositions, and Clusters
13
Modules
• Self-containedreusablecode• Behaviorchangesbasedoninputs• Terraformcode
14
Clusters
Composi=ons
Modules • Founda=on
Composi)ons
• Pre-definedcollec=onsofmodules• Passesparameterstomanymodules• Terraform+Jinja
15
Clusters
Composi=ons • Frame
Modules • Founda=on
Clusters
• Passesparamstoonecomposi=on• Ul=matesourceoftruth• YAML
16
Clusters • Blueprint
Composi=ons • Frame
Modules • Founda=on
For example
• Module• VPCmodule-NATandBas=oninstances,securitygroups,etc• App1module-App1Instances,SQSqueues,S3buckets,subnets• DB1module-RDSinstances,securitygroups• Adminmodule-Instances-configmanagement,monitoring,etc
• Composi=on• Edgecomposi=on-VPC,App1,DB1,Admin• DataProcessingcomposi=on-VPC,App2,DB2,Admin• Deliverycomposi=on-VPC,App3,Admin
17
Analogous to modern Puppet design
• TerraformModules=PuppetModules• Composi=ons=RolesandProfiles• Clusters=ENCandHiera
18
Ops wrapper
• ReadsclusterYAMLvariables• Readscomposi=on(.l.jijna2),writesTerraform(.l)fileswithclustervariablesinjected
19
Demo!
20
The Future
• JenkinsrunsTerraformandcommitsstatefile• WebinterfacetogenerateclusterYAMLfilesforselfservice• Pendingdiscussion:opswrappergeneratesTerraformJSONinsteadofparsingjinja
21
Lessons Learned, Best Prac)ces
• AspringboardforTerraform(opswrapperforus)isinvaluable• TerraformHCL+JinjatemplatesareeasiertowriteandreadthanTerraformJSON
• Make1cluster=1vpc=1environment=1purpose• Reproducibleenvironments• SeparatedTerraformstatefilespercluster
• Versionuserdatainamapvariable• SymlinksharedTerraformfilesinmodules• Separate“common”infrastructurelike-S3buckets,SQS,IAMtoitsowncluster
22
Don’t
• Getimpa=entwithTerraform• Goingunsblazinganduseitinproduc=ononday1• SkipreadingtheTerraformdocs
23
Woohoooo!
• 85%ofourproduc=oninfrastructureismanagedwithTerraform!
24
Ques)ons?
25