Snakes on a Cloud

Python, Boto3, and Getting Things Done

Welcome Buffalo Pythonistas

ryansb@redhat.com

@ryan_sbRyan Scott Brown

Senior Software Engineer

Ansible by Red Hat

Agenda

● Setup & Basics● Talking to Instances● In-Application Use● Ops, Automation, and Hacking the Planet● Testing (if there’s time)

Setup

● Installing● Credentials/Profiles● Hello World

Installing

pip install boto3

pip install awscli #optional

aws configure --profile testbed #optional

Credentials

# ~/.aws/credentials

[testbed]

aws_access_key_id = AKIA…

aws_secret_access_key = Jhw…

Credential Sources

● Instance roles● Environment variables● Access/Secret key pair● Access + secret + session using STS● Credentials files● Assuming cross-account roles

Instance Roles

Your Instance AWS Backplane

GET http://169.254.169.254/…./credentials

Key ID+Secret+Session Token+TTL

Cross-Account Roles

Your Client IAM API

AssumeRole arn:aws:….:SuperAdminKey ID + Secret

Key ID+Secret+Session Token+TTL

Hello World

aws s3 ls

Hello World

>>> import boto3

>>> s3 = boto3.client(‘s3’)

>>> print(s3.list_buckets())

Sessions

● Typically 1 is plenty● If you have multi-region or user requirements, these

handle it well● `session.client(‘service’)` works instead of the default

`boto3.client(‘service’)`

S3 Demo Notebook

Download notebook file

Browser-friendly results

Clients vs. Resources

● Clients are lower-level and usually map 1:1 with APIs● Resources are built around first-class objects● Services always have clients, sometimes have resources● DynamoDB, S3, CloudFormation, and others have both

High Level Services

● App-level services like DynamoDB + S3● Accept user uploads● Generate pre-signed URLs● Store and query key-value data

DynamoDB Demo Notebook

Download Notebook File

Browser-friendly results

Connecting the Planet

● What if you combined dataviz with infra data?● Networkx (graph handling library) can export visual graphs

b

EC2 Demo Notebook

Download notebook file

Browser-friendly results

Hacking the Planet

● Python can be used in combination with other tools● Have a bunch of CloudFormation? No problem

– https://github.com/ryansb/yesterdaytabase

● Ansible? Make a module!● Chef/Puppet? Mix user-data+boto3+Chef Solo/OpsWorks

Extending CloudFormation

● AWS Lambda can be used for custom resources– https://github.com/ryansb/acm-certs-cloudformation

– https://github.com/ryansb/cfn-wrapper-python

● Expose Custom::YourThing interface to templates● Anything boto3 supports, you can add to CloudFormation

Question Time!