level up your cf apps with amazon web...

71
Level Up Your CF Apps with Amazon Web Services Brian Klaas [email protected] @brian_klaas

Upload: others

Post on 25-Jul-2020

1 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

Level Up Your CF Apps with Amazon Web ServicesBrian Klaas

[email protected]

@brian_klaas

Page 2: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda
Page 3: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

Level Up Your CF Apps with Amazon Web ServicesBrian Klaas

[email protected]

@brian_klaas

Page 4: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

Hello

Page 5: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

Hello

Page 6: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

Hello

Page 7: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

Hello

Page 8: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

Hello

Page 9: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

HelloEC2 Lambda

Step Functions

OpsWorks

Cloud Watch

Data Pipeline

VPC IAMDirect

Connect

Cloud Search Elastic Transcoder

SES SNS SQS SWF

Dynamo DB

Elasti Cache

RDS Redshift

Cloud Front Glacier S3 Storage

Gateway

Kinesis

Cloud Trail

Elastic Map Reduce

Elastic Beanstalk

FPS (Payment)

App Stream

WorkMail and Docs

Work Spaces

CognitoMobile

Analytics

Coud HSM

Aurora Athena

Snowball

Xray

Code Commit

Code Deploy

Route 53

Cloud Formation

Elastic Container

ServiceBatch Lightsail Code Build Code

Pipeline

Elasticsearch

AWS Config

API Gateway

Page 10: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

+ =Hello

Page 11: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

Hello

1. Go play in the console 2. Learn about IAM roles and permissions

Things you have to do on your own:

Page 12: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

AWS Playbox github.com/brianklaas/awsplaybox

Page 13: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

S3

Page 14: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

S3

Store all the things.

Page 15: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

S3

99.999999999% durability

Page 16: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

S3

$0.023 per GB stored $0.004 per 10,000 GET $0.005 per 1,000 PUT

$0.10 per GB out after 1GB

Page 17: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

S3

myfiles.s3.amazonaws.com

Bucket

Page 18: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

S3

http://mybucket.s3.amazonaws.com/path/to/file.png

Page 19: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

S3

C:/ s3://

Page 20: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

cffile( variable="fileData", file="s3://somebucket/somefile.txt", action="read" );

cfdirectory( directory="s3://somebucket/someDirectory", action="list" );

S3

Page 21: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

S3

if ( !directoryExists("s3://somebucket.s3.amazonaws.com") ) { perms = [ {group="all", permission="read"}, {id="canonicalIDofYourAWSAccount", permission=“full_control"} ]; cfdirectory( directory="s3://somebucket.s3.amazonaws.com", storeacl=perms, action="create" ); }

fileWrite("s3://somebucket.s3.amazonaws.com/myFile.txt", "#someOutput#");

files = directoryList("s3://somebucket.s3.amazonaws.com");

Page 22: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

S3

IAM Credentials Required

Page 23: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

S3

Inline:

cffile( variable="fileData", file="s3://accessKey:secretKey@somebucket/somefile.txt", action="read" );

Or, in application.cfc:

this.s3.accessKeyId="accessKey"; this.s3.awsSecretKey="secretKey";

Page 24: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

S3

๏ Expire URLs

๏ Change properties on a per-request basis

๏ Upload to S3 from browser

Requires request signing.

Page 25: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

S3

S3 Request Signing Utils

github.com/brianklaas/ctlS3utils

Page 26: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

S3

S3 is storage, not a file system.

Can get basic file info withcfhttp( url="http://bucket.s3.amazonaws.com/filename",

method="head" );

Page 27: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

S3

CF only supports V2 signatures

Can’t use built-in CF support for S3 in new regions since 2014 (Frankfurt, Seoul, Mumbai, Ohio, Canada, London)

Page 28: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

S3

Store all the things.

Page 29: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

DynamoDB

Page 30: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

Dynamo DB

= Hugely scalable, high-write throughput document data store

Page 31: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

Dynamo DB

4. Write3. Set secondary indexes

2. Set primary and sort keys1. Set read/write capacity

Page 32: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

Logging and Monitoring Third–Party Services

Customer uploads Emits event with file info Processes event

File details

Alert

Dynamo DB

Page 33: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

Exam Activity Analytics

Capture learner action Writes batch data Stores batch data

Dynamo DB

?QuickSight

Athena

ElasticSearch

Page 34: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

Demo

List, Put, Scan, Filter from CF

Dynamo DB

github.com/brianklaas/awsplaybox

Page 35: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

Lambda

Page 36: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

Lambda

Page 37: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

Lambda

Page 38: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

Lambda

No server is easier to manage than no server.

Page 39: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

Lambda

= Event–driven computing

Page 40: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

Lambda

<1.5 GB RAM <5 minutes

Page 41: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

Lambda

2. Upload ZIP1. Write a handler function

3. Invoke an event 4. Handler runs

Page 42: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

Lambda

= Microservices infrastructure without having to worry about running containers

or scaling your infrastructure!

Page 43: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

Lambda

Logging and Monitoring Third–Party Services

Page 44: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

Lambda

Logging and Monitoring Third–Party Services

?

Page 45: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

Lambda

Logging and Monitoring Third–Party Services

Customer uploads Emits event with file info Processes event

File details

Alert

Page 46: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

Lambda

Async Image Resizing

Emits event with file info Processes eventHandle customer upload, posts to S3

cffile ( action=“write” file=“s3://somebucket/images.zip” ) Calls “done” http endpoint

Multiple sizes written to S3

Page 47: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

Video Production Workflow

Lambda

Upload to ingest bucket Processes event

Posts to S3

Launches Elastic Transcoder job Puts encoded files

Processes event

Page 48: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

Video Production Workflow (v2)

Lambda

Upload to ingest bucket Processes eventPosts to S3

Launches Elastic Transcoder job Puts encoded files

Processes event

Calls http endpoint Grabs source video Calls transcription API Calls http endpoint Posts to S3

Page 49: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

Demo

Invoking Lambda functions from CF

Lambda

github.com/brianklaas/awsplaybox

Page 50: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

Lambda

= Focus on building apps, not infrastructure

Page 51: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

Step Functions(with some Rekognition)

Page 52: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

Step Functions

Visual serverless orchestration

Page 53: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

Step Functions

Automated, multi-step, asynchronous, serverless workflows in AWS

Page 54: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

Step Functions

Avoid rebuilding the monolith

(Or the microlith)

Page 55: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

Step Functions

Step Functions = “state machine”Amazon States Language (JSON)

https://states-language.net/spec.html

Page 56: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

Step Functions

Page 57: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

Step Functions

Task Sequence

Parallel Branching

Waiting Error Handling

Retries

Page 58: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

Demo

Invoking Step Functions from CF

Step Functions

github.com/brianklaas/awsplaybox

Page 59: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

Step Functions

More on Rekognition in the awsPlaybox

Page 60: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

Step Functions

Automated, multi-step, asynchronous, serverless workflows in AWS

Page 61: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

Step Functions

Use statelint to validate your ASL JSON

https://github.com/awslabs/statelint

Page 62: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

Preparing for Outages

Page 63: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

The Great S3 Outage of 2017

Outages

Page 64: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

Plan for outages.

Outages

(Blame Amazon)

Page 65: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

Batch upload from your servers

Outages

Page 66: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

Use multiple regions

Outages

Page 67: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

Shut off services

Outages

Page 68: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

Have a plan.

Outages

Page 69: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

Go Do!

Page 70: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

AWS Playbox github.com/brianklaas/awsplaybox

Page 71: Level Up Your CF Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/cfSummit2017-LevelUpAppsWithAWS.pdf · = Event–driven computing. Lambda

Please fill out the session survey!

[email protected] @brian_klaas