practical cloud

62

Upload: lynn-langit

Post on 21-Jan-2018

186 views

Category:

Technology


0 download

TRANSCRIPT

• 5 – 15x market share of all others (US market)AWS Dominates

• AWS cloud bills are for ‘old services’ (EC2, RDS)80% EC2++

• First-time cloud projects outcome rate*50% FAIL

Definitions

Vendors

Patterns

Lessons

1

2

3

4

10 Million Executions = ~ $ 500 USD

FilesAWSS3

RDBMSAWSRDS

VMsAWSEC2

Use imageFiles -> “BIG”

Application PLUS OS

Coordinate with Load Balancers

Mature Technology

Use Dockerfiles ->

“small”

Application

Coordinate with Container

Managers

New Technology

Just write code –> “tiny”

Function / Method

Auto-scales

Very New Technology

Use Dockerfiles –> “small”

Application

Coordinate via Container Mgr.

New Technology

100

101

VM Container Lambda

Drive to Lambda – Save MoneyCost

5

50

95

Lambda Container VM

Use VMs – Keep ControlControl

“But why areCompute AND File Storage commoditieson Azure, AWS AND GCP?”

Compute

EC2

Containers

Lambda

Files

S3

Glacier

Data

RDS

DynamoDB

Other

Machine Learning

Kinesis

Serverless

Alpha

Some parts may work

Service may be changed

Service may be discontinued

Beta

Many parts shouldwork

Service may be changed

Year One

Most parts shouldwork

Can include some service integrations

Year Two

All parts shouldwork

Patterns and scripts emerge

Year Three

Service is stable

Tools and partners emerge

$$ $ $$$

Functions• Logic Apps• No Code • Generates JSON

Which Vendor?

Server-based Solutions

Server-based Solutions

High AvailabilityCore Security

Server-based Solutions

High AvailabilitySecurity

Server-based Solutions

High AvailabilitySecurityScalabilityCost Control

Google Compute

Engine

Very fast to start

…globally

Automatically discounted

for sustained use

Easier to size via the ’slider’

Hadoop/Spark Transform

Visualization Client

Data Lake

ANSI SQLExploratory

ANSI SQL Warehouse

Kappa Architecture on the Cloud – Servers?

PySpark ETLGlue

Visualization Client - QuickSight

Data Lake – S3

Explore SQLAthena

MPP SQL Spectrum

AWS

AWS

Beam ETLDataflow

Visualization Client - DataStudio

Data Lake – GCS

Explore SQLBigQuery

MPP SQL BigQuery

Servers / IaaS?

PaaS?

Serverless?

Integration testing?

Orchestration?

Deployment?

“My” Programming

Language ?

Debugging?

Unit testing?

Integration testing?

Orchestration?

Deployment?

AWS X-Ray

Reduce Attack

Surface

Test external

connections

Minimal permissions

Granular policies

Unique credentials

Service Costs

Training Costs

Tooling Costs

Migration Costs

Learning Costs

Service Type Servers (or Containers) Serverless

Compute EC2 Lambda

Files File Services on EC2 S3

SQL on Relational Data RDBMS on EC2 or RDS Athena / Redshift Spectrum

Data Pipeline Kafka cluster on EC2 Kinesis

Machine Learning EMR with Spark ML orHadoop on EC2

Machine Learning API

IoT MQTT Message Broker RabbitMQ on EC2 IoT Broker

NoSQL MongoDB… on EC2 DynamoDB