aws re:invent 2016: how i learned to embrace devops and configure infrastructure at scale (win402)
TRANSCRIPT
© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amjad Hussain
Senior Manager, Amazon EC2
Tim Nicholas
Lead Automation Architect, Xero
12/01/2016
How I learned to embrace DevOps and configure
infrastructure at scale
Amazon EC2 Systems Manager
What to Expect from the Session
• Introduction to EC2 Systems Manager
• Learn about Run Command, State Manager, and
Parameter Store
• How Xero uses Run Command
• Demo!
• FAQs and best-practices
What we heard from you
• Traditional IT tools not built for the
cloud
• Managing resources at scale is difficult
• Lack of visibility into configuration,
granular control
• Multiple vendors; complex licensing
Introducing EC2 Systems Manager
A set of capabilities that enable automated configuration and
ongoing management of systems at scale, across all your
Windows and Linux workloads, running in Amazon EC2 or
on-premises
Systems Manager Capabilities
Run Command Maintenance
Window
Inventory
State Manager Parameter Store
Patch Manager
Automation
Deploy, Configure,
and Administer
Track and
UpdateShared
Capabilities
Run Command
Run Command
• Execution of administrative tasks
• Improve security posture – no need to SSH or RDP
• Delegated access control
• Customizable and flexible
• Get notified on the status of your commands
• Control the rate at which you send commands for scale
Sending a command
aws ssm send-command
--document-name AWS-RunPowerShellScript
--instance-id i-1234567
--parameters commands=“mkdir C:\Demo”
--service-role-arn <my-service-role>
-- notification-config NotificationArn=<my-topic-
arn>,NotificationEvents=“Success”,NotificationType=“Command”
Remotely create a directory on an instance and notify via
SNS when it completes
Run Command – Getting started
• Instance: Setup agent, AWS Identity & Access
Management (IAM) role on your instance
• Document: Author your intent
• Command and Command Invocation
• Plugins: In-guest actions that perform tasks
• Status and output: Granular results
Run Command – Scale
• Send a command based on a tag query
• Velocity control and error handling
aws ssm send-command --document-name <value> --targets
“Key=tag:ServerRole;Values=WebFrontEnd” […]
aws ssm send-command --max-concurrency 10 …
aws ssm send-command --max-errors 10 …
Setting up your instances
• Single light-weight agent, cross-platform
• SSM agent is open source, written in Go
• Health status via DescribeInstanceInformation
• On-demand agent update
• Hybrid support
Finding out which instances are heartbeating
D:\Users\amjadhu>aws ssm describe-instance-information
{
"InstanceInformationList": [
{
"IsLatestVersion": false,
"PingStatus": "Online",
"InstanceId": "i-c6d69773",
"ResourceType": "EC2Instance",
"AgentVersion": "3.17.1032",
"PlatformVersion": "6.2.9200",
"PlatformName": "Windows Server 2012 Standard",
"PlatformType": "Windows",
"LastPingDateTime": 1477203028.78
},
Setting up the agent for on-premises
• One-time setup to register
on-premises servers
• Consistent experience
• Identified by mi-*
• One-time setup to register
on-premises servers
• Consistent experience
• Identified by mi-*
Customizing commands
• Documents: A common way of authoring across EC2
Systems Manager
• Parameters: Allows passing in run-time values
• JSON schema, allows editing and versioning
• Sharing with accounts
• Amazon published documents (begin with AWS-*)
Document content
{
"schemaVersion": "2.0",
"description": “Simple script execution",
"mainSteps": [
{
"action": "aws:runPowerShellScript",
"name": “runMyScript",
"inputs": {
"runCommand": “.\myScript.ps1",
"workingDirectory": "C:\Temp",
"timeoutSeconds": "10"
}
}
]
}
Sharing Documents
• Share with other accounts
• Cross-account scenarios
How Xero uses Run Command
Xero – Run Command
Beautiful cloud-
based
accounting software
Connecting people with the right numbers
anytime, anywhere, on any device
Beautiful accounting software
Xero – Run Command
1,500+Staff globally
862kSubscribers globally
$303msub revenue FY16
All figures shown are in NZD
2 yearsAWS design and build
6 monthsAWS service migration
Xero – Operational Challenges
• Host discovery
• Dynamic, disposable servers
• Increase in host count
• Integration with pipeline tools
• CI/CD tooling
• Chat bots
• Lambda
• Network isolation
• Production Servers vs Pipeline tools (git, CI/CD)
• Multiple AWS accounts
• Production Servers vs operations/development team workstations
prox
yWAF
prox
yWAF
internetcustomer
sadmin
Prod TPZServices
TPZ
Prod App VPC Services App VPC
Xero – Network Separation
Xero – Run Command Use Cases
• Validation of .Net application configuration - From CI
• Reloading application pools - via CI
• Enabling services on a sample of machines in an
ASG via AWS Lambda
• PowerShell modules for interactive investigation
Xero – Demo!
PowerShell modules for interactive
investigation
State Manager
State Manager
• Maintain consistent state of instances
• Reapply to keep instances from drifting
• Easily view status of configuration changes
• Define schedule – ad hoc, periodic
• Track aggregate status for your fleet
State Manager – Getting started
• Document: Author your intent
• Target: Instances or tag queries
• Association: Binding between a document and a
target
• Schedule: When to apply your association
• Status: Check the state of your association at an
aggregate or instance level
Creating an Association
aws ssm create-association
--document-name WebServerDocument
--document-version \$DEFAULT
--schedule-expression cron(0 */30 * * * ? *)
--targets “Key=tag:Name;Values=WebServer”
--output-location "{ \"S3Location\": { \"OutputS3Region\": \“us-east-1\",
\"OutputS3BucketName\": \“MyBucket\", \"OutputS3KeyPrefix\": \“MyPrefix\" } }“
Configures all instances that match the tag query and reapplies every
30 minutes
Parameter Store
Parameter Store
• Centrally store and find config data
• Repeatable, automatable management (e.g. SQL
connection strings)
• Granular access control – view, use and edit values
• Encrypt sensitive data using your own AWS KMS keys
Parameter Store – Getting started
• Parameter: Key-value pair
• Secure Strings: Encrypt sensitive parameters with your
own KMS or default account encryption key
• Reuse: In Documents and easily reference at runtime
across EC2 Systems Manager using {{ssm:parameter-
name}}
• Access Control: Create an IAM policy to control access
to specific parameter
Creating and using a parameter
aws ssm put-parameter
--name mycommand
--type string
--value “dir C:\Users”
aws ssm send-command
--name AWS-RunPowerShellScript
--parameters commands=[“echo {{ssm:mycommand}}”]
--target Key=tag:Name,Values=WebServer
Demo!
Best-practices and FAQs
• What OS platforms are supported?
• Update your SSM agent today to get started!
• What ports or network access do my instances need?
• Is there anything different to set up on-premises servers?
• Use notifications, velocity control
• For disruptive actions, use Run Command with Maintenance
Window
• Fine-grained access control through IAM policies on resources (e.g.
documents)
• Customize configuration with idempotent scripts for State Manager
Your Feedback is Important!
• These services are available today
• Learn more at https://aws.amazon.com/ec2/run-
command/
• Technical documentation at
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/
run-command.html
• Please send your feedback, improvements, requests to
Thank you!
Remember to complete
your evaluations!
Related Sessions
WIN401 - How to Manage Inventory, Patching, and System
Images for Your Hybrid Cloud with AWS Management
Capabilities