(dvo301) aws opsworks under the hood
TRANSCRIPT
© 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Thomas Kaschwig and Jonathan Weiss Amazon Web Services
October 2015
DVO301
AWS OpsWorks Under the Hood
Agenda
• Chef in AWS OpsWorks
• Chef 12.x integration
• Windows support
• Amazon ECS integration + Demo
• Amazon EC2 import and on-premises servers + Demo
AWS OpsWorks architecture
AWS OpsWorks
Backend
AWS OpsWorks
Agent
Amazon EC2, Amazon EBS, Amazon RDS,
Amazon VPC, Elastic Load Balancing,
Amazon ECS, Auto Scaling, auto-healing,...
On-instance execution via
Chef client/zero
Command
JSON
Command
Log+Status
Chef integration
• Supports Chef 11.10 and Chef 12.x
• Built-in convenience cookbooks / bring your own
• Chef run is triggered by lifecycle event firing:
push vs. pull
• Event comes with stack state JSON
Setup event
• Sent when instance boots
• Includes deploy event
• Use for initial installation of
software & services
Configure event
• Sent to all instances when any
instance enters or leaves online
state
• Use for making sure the
configuration is up-to-date
Custom recipes – Chef 11.10
Single Chef run with a merged run_list
AWS
OpsWorks
Recipes
Your
Custom
Recipes
Combined Chef run using the same environment
Custom recipes – Chef 12.x
Two separate Chef runs and thus separate run_lists
AWS
OpsWorks
Recipes
Your
Custom
Recipes
Internal environment Customer environment
chef-client 12.x
Chef 12.x environment
Customer-only Chef run:
• Empty run_list
• No OpsWorks cookbooks polluting your namespace
• node[:opsworks] no longer present, use search instead
Search
Stack state JSON available through search
search(:node, “name:web1”)
search(:node, “name:web*”)
Attributes generated on nodes are not available
Search
appserver = search(:node, "role:php-app").first
Chef::Log.info(”Private IP: #{appserver[:private_ip]}")
Exposes: hostname/FQDN, IP/DNS, private IP/DNS,
instance type, AMI ID, AZ, …
Roles
appserver = search(:node, "role:php-app").first
Chef::Log.info(”Private IP: #{appserver[:private_ip]}")
AWS OpsWorks layers mapped as roles
Search
search(:aws_opsworks_app, "name:myapp")
search(:aws_opsworks_app, ”deploy:true")
search(:aws_opsworks_layer, "name:rails*")
search(:aws_opsworks_rds_db_instance)
search(:aws_opsworks_volume)
search(:aws_opsworks_ecs_cluster)
search(:aws_opsworks_elastic_load_balancer)
search(:aws_opsworks_user)
http://docs.aws.amazon.com/opsworks/latest/userguide/attributes-json-windows.html
Data Bags
Define in custom JSON
{"opsworks": {
"data_bags": {"bag_name1": {"item_name1: {
"key1" : “value1”,"key2" : “value2”,...
}},"bag_name2": {"item_name1": {
"key1" : “value1”,"key2" : “value2”,...
}},...
}}
}
{ "opsworks": {
"data_bags": {"myapp": {"mysql": {
"username": "default-user","password": "default-pass"
}}
}}
}
mything = data_bag_item("myapp", "mysql")Chef::Log.info("username: #{mything['username']}")
Recipe
Encrypted Data Bags
Alternative handling:
• App environment variables
• Upload encrypted JSON to Amazon S3
• Leverage IAM roles for Amazon EC2 in recipe
App environment variables
user = ENV[“payment_provider_user“]password = ENV[“payment_provider_password“]
PaymentGateway.new(user, password)
Windows support
Supported as default AMIs:
• Windows Server 2012 R2 Standard
• Windows Server 2012 R2 SQL Server Express
• Windows Server 2012 R2 SQL Server Standard
• Windows Server 2012 R2 SQL Server Web
Amazon EC2 Container Service (Amazon ECS)
• Highly scalable and fast container management service
that makes it easy to run and manage Docker containers
on a cluster of Amazon EC2 instances
• Lets you launch and manage container-enabled
applications with simple API calls
• Monitor the state of your cluster with a centralized
service
Amazon ECS integration in AWS OpsWorks
• Streamlined ECS container instances provisioning and
management
• ECS container instances operating system and package
updates
• User permission management
• ECS container instance performance monitoring
• Amazon EBS volume management
• Public and Elastic IP address management
• Security group management
Motivation
• Manage servers not running on Amazon EC2 with
AWS OpsWorks
• Enable hybrid environments
• Run development and test stacks on EC2 before
deploying to your on-premises fleet
• Scale out from your on-premises server infrastructure to
Amazon EC2
Support for on-premises servers (I)
• Deploy and operate applications on any server with
Internet connection including physical hardware and
VMs in your datacenter
• AWS OpsWorks can update operating systems and
software across your entire fleet
• AWS OpsWorks can run scripts or Chef recipes for you
on your entire fleet
• You can control who can run scripts and you are able to
view a history of each script that has been run
Support for on-premises servers (II)
• Manage operating system users and ssh/sudo access
• Amazon CloudWatch metrics for CPU, memory and load
for your on-premises servers
• Pricing: $0.02/hour per registered on-premises server
Support for existing Amazon EC2 instances
• Installs the AWS OpsWorks agent on already running
Amazon EC2 instances
• Registers the instances to existing OpsWorks stacks
• Assigns the instances to one or multiple OpsWorks
layers
• Benefit from AWS OpsWorks management features
• No additional charges
Registration lifecycle
AWS CLIaws opsworks register
Registering
RegisteredAssigning
Online
Running
Setup
Running
Setup