Download - (DEV301) Automating AWS with the AWS CLI
![Page 1: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/1.jpg)
© 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
James Saryerwinnie
October 2015
Automating AWS with the AWS CLI
DEV301
![Page 2: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/2.jpg)
AWS Command Line Interface
Unified tool to manage your AWS services
![Page 3: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/3.jpg)
$ git show 1.0.0tag 1.0.0Tagger: James Saryerwinnie <[email protected]>Date: Mon Sep 2 18:38:51 2013 -0700
Tagging 1.0.0 release.
commit 7fbdac0b19ea9dcc0380242068af20ef425ac5c3Merge: 6f76721 469bab6Author: James Saryerwinnie <[email protected]>Date: Mon Sep 2 18:38:51 2013 -0700
Merge branch 'release-1.0.0'
* release-1.0.0:Bumping version to 1.0.0
![Page 4: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/4.jpg)
Interactive
![Page 5: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/5.jpg)
Shell scripts#!/bin/bash
import_key_pair() {echo -n "Would you like to import ~/.ssh/id_rsa.pub? [y/N]: "read confirmationif [[ "$confirmation" != "y" ]]then
return
fi
aws ec2 import-key-pair \
--key-name id_rsa \
--public-key-material file://~/.ssh/id_rsa.pub}
create_instance_profile() {echo -n "Would you like to create an IAM instance profile? [y/N]: "read confirmationif [[ "$confirmation" != "y" ]]; then
return
fi
aws iam create-role --role-name dev-ec2-instance \
--assume-role-policy-document "$TRUST_POLICY" || errexit "Could not create Role"
# Use a managed policy
policies=$(aws iam list-policies --scope AWS)
admin_policy_arn=$(jp -u \
"Policies[?PolicyName=='AdministratorAccess'].Arn | [0]" <<< "$policies")
aws iam attach-role-policy \
--role-name dev-ec2-instance \
--policy-arn "$admin_policy_arn" || errexit "Could not attach role policy"
# Then we need to create an instance profile from the role.
aws iam create-instance-profile \
--instance-profile-name dev-ec2-instance || \
errexit "Could not create instance profile."# And add it to the role
aws iam add-role-to-instance-profile \
--role-name dev-ec2-instance \
--instance-profile-name dev-ec2-instance || \
errexit "Could not add role to instance profile."}
compute_key_fingerprint() {# Computes the fingerprint of a public SSH key given a private
# RSA key. This can be used to compare against the output given
# from aws ec2 describe-key-pair.
openssl pkey -in ~/.ssh/id_rsa -pubout -outform DER | \
openssl md5 -c | \
cut -d = -f 2 | \
tr -d '[:space:]'}
do_setup() {echo "Checking for required resources..."echo ""# 1. Check if a security group is found for
# both windows and non-windows tags.
# If not, we'll eventually give the option to
# configure this.
if resource_exists "aws ec2 describe-security-groups \--filter Name=tag:dev-ec2-instance,Values=non-windows"; then
echo "Security groups exists."else
echo "Security group not found."fi
echo ""
# 2. Make sure the keypair is imported.
if [[ ! -f ~/.ssh/id_rsa ]]; then
echo "Missing ~/.ssh/id_rsa key pair."elif has_new_enough_openssl; then
fingerprint=$(compute_key_fingerprint ~/.ssh/id_rsa)
if resource_exists "aws ec2 describe-key-pairs \--filter Name=fingerprint,Values=$fingerprint"; then
echo "Key pair exists."else
echo "~/.ssh/id_rsa key pair does not appear to be imported."import_key_pair
fi
else
echo "Can't check if SSH key has been imported."echo "You need at least openssl 1.0.0 that has a \"pkey\" command."echo "Please upgrade your version of openssl."
fi
echo ""
# 3. Check that they have an IAM role called dev-ec2-instance.
# There is no server side filter for this like we have with the EC2
# APIs, so we have to manually use a --query option for us.
if resource_exists "aws iam list-instance-profiles" \
"length(InstanceProfiles[?InstanceProfileName=='dev-ec2-instance'])"; then
echo "Instance profile exists."else
echo "Missing IAM instance profile 'dev-ec2-instance'"create_instance_profile
fi
echo ""}
do_setup
![Page 6: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/6.jpg)
aws s3 lsApplicationwith an AWS SDK
![Page 7: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/7.jpg)
aws s3 lsApplicationwith an AWS SDK
Shell Scripts – what we’ll cover
![Page 8: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/8.jpg)
A good shell script
• < 100 SLOC
• Commands in sequence
• Piping input/output
• Simple domain logic
![Page 9: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/9.jpg)
• Familiar with AWS CLI
• Familiar with bash
• Familiar with AWS
Prerequisites
![Page 10: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/10.jpg)
$ aws ec2 describe-instances --filters Name=architecture,Values=x86_64
$ du $(ls | grep cli) | sort -nr | tail -2
Amazon S3Amazon EC2
Prerequisites
![Page 11: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/11.jpg)
Getting up to speed
• Becoming a Command Line Expert with the AWS CLI
• Advanced Usage of the AWS CLI
• Installation
• Configuration
![Page 12: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/12.jpg)
$
We will be in the terminal…
![Page 13: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/13.jpg)
$
We will be in the terminal…a lot
![Page 14: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/14.jpg)
Common patterns
![Page 15: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/15.jpg)
Main patterns
• Single output to a single command
• Map list output to N AWS CLI commands
• Storing JSON documents and querying later
• Resource exists check
![Page 16: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/16.jpg)
Main patterns
• Single output to a single command
• Map list output to N AWS CLI commands
• Storing JSON documents and querying later
• Resource exists check
![Page 17: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/17.jpg)
Output to a single parameter
![Page 18: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/18.jpg)
Output to a single parameter
$ aws ec2 describe…
{
"Reservations": [
{
"Groups": [
{
"GroupId": "sg-abc",
"GroupName": "Windows"
}
],
"Instances": [
{
"AmiLaunchIndex": 0,
"Architecture": "x86_64",
"BlockDeviceMappings": [
{
"DeviceName": "/dev/sda1",
"Ebs": {
"AttachTime": "2015-01-29T21:22:17.000Z",
"DeleteOnTermination": true,
"Status": "attached",
$ aws ec2 run… --foo <here>
![Page 19: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/19.jpg)
Output to a single parameter
aws ec2 create-tags --tags Key=purpose,Value=dev \
--resources $(aws ec2 run-instances \
--image-id ami-12345 \
--query Instances[].InstanceId \
--output text)
![Page 20: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/20.jpg)
Output to a single parameter
aws ec2 create-tags --tags Key=purpose,Value=dev \
--resources $(aws ec2 run-instances \
--image-id ami-12345 \
--query Instances[].InstanceId \
--output text)
Error handling
![Page 21: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/21.jpg)
Output to a single parameter
instance_ids=$(aws ec2 run-instances \
--image-id ami-12345 \
--query Instances[].InstanceId \
--output text) || errexit "Could not run instance"aws ec2 create-tags \
--tags Key=purpose,Value=dev \
--resources "$(instance_ids)" || errexit "<errmsg>"
![Page 22: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/22.jpg)
Output to a single parameter
instance_ids=$(aws ec2 run-instances \
--image-id ami-12345 \
--query Instances[].InstanceId \
--output text) || errexit "Could not run instance"aws ec2 create-tags \
--tags Key=purpose,Value=dev \
--resources "$(instance_ids)" || errexit "<errmsg>"
![Page 23: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/23.jpg)
Output to a single parameter
instance_ids=$(aws ec2 run-instances \
--image-id ami-12345 \
--query Instances[].InstanceId \
--output text) || errexit "Could not run instance"aws ec2 create-tags \
--tags Key=purpose,Value=dev \
--resources "$(instance_ids)" || errexit "<errmsg>"
![Page 24: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/24.jpg)
Output to a single parameter
• Used to map one part of the output to one part of the input
• Use --query and --output text
• Make sure to handle the error case before using the value
![Page 25: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/25.jpg)
Main Patterns
• Single output to a single command
• Map list output to N AWS CLI commands
• Storing JSON documents and querying later
• Resource exists check
![Page 26: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/26.jpg)
Mapping one output to N AWS calls
![Page 27: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/27.jpg)
Mapping one output to N AWS calls
$ aws iam list…
[
"a",
"b",
”c",
"d",
"e"
]
$ aws iam … --foo <here>
$ aws iam … --foo <here>
$ aws iam … --foo <here>
$ aws iam … --foo <here>
$ aws iam … --foo <here>
![Page 28: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/28.jpg)
Mapping one output to N AWS calls
for name in $(aws iam list-users \
--query "Users[].[UserName]" --output text); do
aws iam delete-user --user-name "$name"done
![Page 29: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/29.jpg)
Mapping one output to N AWS calls
for name in $(aws iam list-users \
--query "Users[].[UserName]" --output text); do
aws iam delete-user --user-name "$name"done
User1User2User3User4User5
![Page 30: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/30.jpg)
Mapping one output to N AWS calls
for name in $(aws iam list-users \
--query "Users[].[UserName]" --output text); do
aws iam delete-user --user-name "$name"done
![Page 31: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/31.jpg)
Mapping one output to N AWS calls
aws iam list-users --query "Users[].[UserName]" --output text |xargs -I {} -P 10 aws iam delete-user --user-name "{}"
![Page 32: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/32.jpg)
Mapping one output to N AWS calls
aws iam list-users --query "Users[].[UserName]" --output text |xargs -I {} -P 10 aws iam delete-user --user-name "{}"
![Page 33: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/33.jpg)
Mapping one output to N AWS calls
aws iam list-users --query "Users[].[UserName]" --output text |xargs -I {} -P 10 aws iam delete-user --user-name "{}"
Parallel
![Page 34: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/34.jpg)
Mapping one output to N AWS calls
\-+= 72730 james -bash\-+- 76343 james xargs -I {} –P 9 aws iam delete-user --user-name {}
|--- 76348 james aws iam delete-user --user-name user1|--- 76349 james aws iam delete-user --user-name user2|--- 76350 james aws iam delete-user --user-name user3|--- 76351 james aws iam delete-user --user-name user4|--- 76352 james aws iam delete-user --user-name user5|--- 76353 james aws iam delete-user --user-name user6|--- 76354 james aws iam delete-user --user-name user7|--- 76355 james aws iam delete-user --user-name user8\--- 76356 james aws iam delete-user --user-name user9
![Page 35: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/35.jpg)
Mapping one output to N AWS calls
• Use a for loop for functions
• If you’re using xargs, use -I {}
• Use xargs –P N parallel execution
• Use “.[UserName]” instead of “.UserName” to get
newline separated output..
• This works because nothing is written to stdout on error
![Page 36: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/36.jpg)
Demo
![Page 37: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/37.jpg)
Main patterns
Single output to a single command
Map list output to N AWS CLI commands
• Storing JSON documents and querying later
• Resource exists check
![Page 38: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/38.jpg)
Saving the entire output for querying later
{"AmiLaunchIndex": 0,"Architecture": "x86_64","BlockDeviceMappings": [{"DeviceName": "/dev/sda1","Ebs": {"AttachTime": "2015-01-29T21:22:17.000Z","DeleteOnTermination": true,"Status": "attached","VolumeId": "vol-12345"
}},{"DeviceName": "xvdg","Ebs": {"AttachTime": "2015-09-17T19:14:27.000Z","DeleteOnTermination": false,"Status": "attached","VolumeId": "vol-12345"
}}
],"ClientToken": "12345","EbsOptimized": false,"Hypervisor": "xen","IamInstanceProfile": {"Arn": "arn:aws:iam::12345","Id": "12345"
},"ImageId": "ami-12345","InstanceId": "i-12345","InstanceType": "m3.large","KeyName": "id_rsa","LaunchTime": "2015-09-17T17:24:13.000Z","Monitoring": {"State": "disabled"
},}
instance =
![Page 39: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/39.jpg)
Saving the entire output for querying later
{"AmiLaunchIndex": 0,"Architecture": "x86_64","BlockDeviceMappings": [{"DeviceName": "/dev/sda1","Ebs": {"AttachTime": "2015-01-29T21:22:17.000Z","DeleteOnTermination": true,"Status": "attached","VolumeId": "vol-12345"
}},{"DeviceName": "xvdg","Ebs": {"AttachTime": "2015-09-17T19:14:27.000Z","DeleteOnTermination": false,"Status": "attached","VolumeId": "vol-12345"
}}
],"ClientToken": "12345","EbsOptimized": false,"Hypervisor": "xen","IamInstanceProfile": {"Arn": "arn:aws:iam::12345","Id": "12345"
},"ImageId": "ami-12345","InstanceId": "i-12345","InstanceType": "m3.large","KeyName": "id_rsa","LaunchTime": "2015-09-17T17:24:13.000Z","Monitoring": {"State": "disabled"
},}
instance = query $instance \BlockDeviceMappings[0].VolumeId
![Page 40: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/40.jpg)
Saving the entire output for querying later
{"AmiLaunchIndex": 0,"Architecture": "x86_64","BlockDeviceMappings": [{"DeviceName": "/dev/sda1","Ebs": {"AttachTime": "2015-01-29T21:22:17.000Z","DeleteOnTermination": true,"Status": "attached","VolumeId": "vol-12345"
}},{"DeviceName": "xvdg","Ebs": {"AttachTime": "2015-09-17T19:14:27.000Z","DeleteOnTermination": false,"Status": "attached","VolumeId": "vol-12345"
}}
],"ClientToken": "12345","EbsOptimized": false,"Hypervisor": "xen","IamInstanceProfile": {"Arn": "arn:aws:iam::12345","Id": "12345"
},"ImageId": "ami-12345","InstanceId": "i-12345","InstanceType": "m3.large","KeyName": "id_rsa","LaunchTime": "2015-09-17T17:24:13.000Z","Monitoring": {"State": "disabled"
},}
instance =
query $instance ImageId
query $instance \BlockDeviceMappings[0].VolumeId
![Page 41: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/41.jpg)
Saving the entire output for querying later
{"AmiLaunchIndex": 0,"Architecture": "x86_64","BlockDeviceMappings": [{"DeviceName": "/dev/sda1","Ebs": {"AttachTime": "2015-01-29T21:22:17.000Z","DeleteOnTermination": true,"Status": "attached","VolumeId": "vol-12345"
}},{"DeviceName": "xvdg","Ebs": {"AttachTime": "2015-09-17T19:14:27.000Z","DeleteOnTermination": false,"Status": "attached","VolumeId": "vol-12345"
}}
],"ClientToken": "12345","EbsOptimized": false,"Hypervisor": "xen","IamInstanceProfile": {"Arn": "arn:aws:iam::12345","Id": "12345"
},"ImageId": "ami-12345","InstanceId": "i-12345","InstanceType": "m3.large","KeyName": "id_rsa","LaunchTime": "2015-09-17T17:24:13.000Z","Monitoring": {"State": "disabled"
},}
instance =
query $instance InstanceId
query $instance ImageId
query $instance \BlockDeviceMappings[0].VolumeId
![Page 42: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/42.jpg)
jp – The JMESPath CLI
• Same language as --query
• Cross platform executable
• https://github.com/jmespath/jp
![Page 43: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/43.jpg)
jp – The JMESPath CLI
• Same language as --query
• Cross platform executable
• https://github.com/jmespath/jp
Mac OS Xbrew tap jmespath/jmespathbrew install jp
![Page 44: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/44.jpg)
Saving the entire output for querying later
instance=$(aws run-instance --image-id ami-1234 \
--query Instances[0])
query() {jp -u "$2" <<<"$1"
}
instance_id=$(query "$instance" InstanceId)
state_name=$(query "$instance" State.Name)
instance_name=$(query "$instance" "Tags[?Key=='Name'].Value | [0]")
![Page 45: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/45.jpg)
Saving the entire output for querying later
instance=$(aws run-instance --image-id ami-1234 \
--query Instances[0])
query() {jp -u "$2" <<<"$1"
}
instance_id=$(query "$instance" InstanceId)
state_name=$(query "$instance" State.Name)
instance_name=$(query "$instance" "Tags[?Key=='Name'].Value | [0]")
instance=
{"InstanceId": ”i-12345", ... }
![Page 46: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/46.jpg)
Saving the entire output for querying later
instance=$(aws run-instance --image-id ami-1234 \
--query Instances[0])
query() {jp -u "$2" <<<"$1"
}
instance_id=$(query "$instance" InstanceId)
state_name=$(query "$instance" State.Name)
instance_name=$(query "$instance" "Tags[?Key=='Name'].Value | [0]")
![Page 47: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/47.jpg)
Saving the entire output for querying later
instance=$(aws run-instance --image-id ami-1234 \
--query Instances[0])
query() {jp -u "$2" <<<"$1"
}
instance_id=$(query "$instance" InstanceId)
state_name=$(query "$instance" State.Name)
instance_name=$(query "$instance" "Tags[?Key=='Name'].Value | [0]")
![Page 48: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/48.jpg)
Saving the entire output for querying later
instance=$(aws run-instance --image-id ami-1234 \
--query Instances[0])
query() {jp -u "$2" <<<"$1"
}
instance_id=$(query "$instance" InstanceId)
state_name=$(query "$instance" State.Name)
instance_name=$(query "$instance" "Tags[?Key=='Name'].Value | [0]")
![Page 49: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/49.jpg)
Saving the entire output for querying later
1. Use JSON output, not text, --output json
2. Store the entire thing as a variable, or write to a temp
file if output size is a concern.
3. Use the “jp” command to filter down results as needed.
![Page 50: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/50.jpg)
Main Patterns
Single output to a single command
Map list output to N AWS CLI commands
Storing JSON documents and querying later
• Resource exists check
![Page 51: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/51.jpg)
resource_exists() {
}
1. Determine command to run with query
2. Run command and check errors
3. Check length of query result
![Page 52: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/52.jpg)
resource_exists() {local cmdlocal num_matches
if [[ -z "$2" ]]; then
cmd="$1 --query length(*[0])"else
cmd="$1 --query $2"fi
}
2. Run command and check errors
3. Check length of query result
![Page 53: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/53.jpg)
resource_exists() {
num_matches=$($command)
if [[ "$?" -ne 0 ]]; then
echo "Could not check if resource exists, exiting."exit 2
fi
}
3. Check length of query result
1. Determine command to run with query
![Page 54: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/54.jpg)
resource_exists() {
if [[ "$num_matches" -gt 0 ]]; then
# RC of 0 mean the resource exists, success.
return 0else
return 1fi
}
1. Determine command to run with query
2. Run command and check errors
![Page 55: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/55.jpg)
Usage: Server-side filtering
describe_groups="aws ec2 describe-security-groups"if resource_exists "$describe_groups \--filter Name=tag:dev-ec2-instance,Values=linux"; then
echo "Security groups exists."else
echo "Security group not found."fi
![Page 56: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/56.jpg)
Usage: Client-side filtering
list_profiles="aws iam list-instance-profiles"if resource_exists "$list_profiles" \
"InstanceProfiles[?InstanceProfileName=='dev-ec2-instance']" then
echo "Instance profile exists."else
echo "Missing IAM instance profile 'dev-ec2-instance'"create_instance_profile
fi
![Page 57: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/57.jpg)
Resource exists
• Check if a resource exists by filtering down a list and
checking if the length is greater than 0.
• Use server side filtering when possible
• Use JMESPath query for client side filtering
![Page 58: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/58.jpg)
Demo
![Page 59: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/59.jpg)
Common Patterns
Single output to a single command
Map list output to N AWS CLI commands
Storing JSON documents and querying later
Resource exists check
![Page 60: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/60.jpg)
Deployment tips
![Page 61: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/61.jpg)
# * * * * * command to execute# │ │ │ │ │# │ │ │ │ │# │ │ │ │ └───── day of week (0 – 6)# │ │ │ └────────── month (1 - 12)# │ │ └─────────────── day of month (1 - 31)# │ └──────────────────── hour (0 - 23)# └───────────────────────── min (0 - 59)
Cron
![Page 62: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/62.jpg)
$ whoami
$HOME/.aws/config
$HOME/.aws/credentials
export AWS_CONFIG_FILE
export AWS_SHARED_CREDENTIALS_FILE
![Page 63: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/63.jpg)
$ whoami
$HOME/.aws/config
$HOME/.aws/credentials
export AWS_CONFIG_FILE
export AWS_SHARED_CREDENTIALS_FILE
• Explicitly set AWS_CONFIG_FILE/AWS_SHARED_CREDENTIALS_FILE• Ensure user running cron job has ~/.aws/config ~/.aws/credentials
![Page 64: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/64.jpg)
aws configure set region \$(curl -s http://169.254.169.254/latest/dynamic/instance-identity/document \| jp -u 'region')
Configure a region
![Page 65: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/65.jpg)
Summary
• Common patterns
• Shell script examples
• Deployment
• Tools
![Page 66: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/66.jpg)
Remember to complete
your evaluations!
![Page 67: (DEV301) Automating AWS with the AWS CLI](https://reader034.vdocument.in/reader034/viewer/2022050613/5882389f1a28ab31228b5921/html5/thumbnails/67.jpg)
Thank you!• User guide: http://docs.aws.amazon.com/cli/latest/userguide/
• Reference docs: http://docs.aws.amazon.com/cli/latest/reference/
• JMESPath: http://jmespath.org/
• re:Invent 2014 - https://www.youtube.com/watch?v=vP56l7qThNs
• re:Invent 2013 - https://www.youtube.com/watch?v=qiPt1NoyZm0