new launch! introducing aws snowball edge and aws snowmobile

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Colin Lazier, General Manager Amazon Glacier, AWS Snowball, & AWS Snowmobile

Jay Littlepage, VP, Infrastructure & Operations, DigitalGlobe

Rich Ridolfo, Sr. Dir, Operations, Philips

December 1, 2016

NEW LAUNCH!

IntroducingAWS Snowmobile and AWS Snowball Edge

Migrating large scale data is challenging

Speed

$ Cost

Logistics

Security

The AWS data transport solution

AWS Snowball—petabyte scale data transport

• Rugged 8.5G impact case

• E-ink shipping label

• Data encryption end-to-end

• 50 TB capacity/10G network

E-ink shipping

label

RE:INVENT 2015 LAUNCH

How Snowball moves data into and out of AWS

Create

a job

Connect the

Snowball

Copy data to

the Snowball

Your data

moved to S3

In transit to you Delivered to you Delivered to AWS At AWS

Job created Job completed

Snowball to date

SPEED

COSTLOGISTICS

SECURITY • 80TB Snowball

• HIPAA Compliance

• Regions:

N. Virginia, Ohio, Oregon,

N. California, GovCoud,

Ireland, Frankfurt, Sydney

• Client updates

• HDFS Ingest

• S3 Compatible Endpoint

100x

Challenges at

extreme

scale

80TB

100PB

Exabyte

Increased logistics complexity Increased time for cycles

Snowmobile broad strokes

• 100 Petabytes per job

• Truck with 45 foot container

• 68,000 pounds

• Up to 500Gbps

• Site Survey

• Dispatch

• Fill

• Return for Ingest

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Jay Littlepage

DigitalGlobe

Q3 2016 IT All Hands - August 2016

Worldview3 location: 1480 km west of Los Angeles (Lat/Lon 32.58 N/133.96 W)

slant range from Las Vegas: 1950 km at an azimuth of 263 deg (W)

Worldview3 location: 1480 km west of Los Angeles (Lat/Lon 32.58 N/133.96 W)

slant range from Las Vegas: 1950 km at an azimuth of 263 deg (W)

Dealing with “Data Boulders from Space”

• A single image from

WorldView3 is 12GB.

• We are imaging

24x7, adding 85TB

of imagery to our

archive, every day.

0

200

400

600

800

1,000

1,200

2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016

Are

a C

ollecti

on

Per

Year

(Millio

ns S

QK

M)

Year

WV3

WV2

WV1

GE1

QB

IKONOS

“Seeing a Better World” for 16 years, with more sensors at higher resolution,

adds up to a data management challenge.

Release the images!

How? Snowball!

$Tape

Archive

DMP

Service Nodes

First stage booster: push interesting data sets to AWS via

Snowball

$$

Staging

Cache

Fast

Cache

$$$

. . . DMP

Data Movers

Image Processing

Client

Data

Mover

Where?/fast/cache/location

Client

Clearly, we needed a bigger

Snowball. One visible from space.

DMP

Service Nodes

Second stage booster: push entire wideband archive to Glacier vault

via Snowmobile. No more tape robots!

$$

Staging

Cache

Fast

Cache

$$$

. . . DMP

Data Movers

Image Processing

Client

Data

Mover

Where?/fast/cache/location

Data

Mover

Snowmobile

Lessons we are learning with Snowmobile

• This is not plug and play (yet)

• Pre-planning and coordination is

critical.

• Understand payloads

beforehand – a lot of tuning to

optimized for our large files (i.e.

data boulders from space)

• The secret sauce: Compute

intensive translation between

legacy file systems like ours and

the data center on wheels

On orbit: Cloud-centric data management.

Where?

https://s3.amazon.com/bucket/obj

Location

Service / Stage

Instances

Where?

https://s3.amazon.com/bucket/obj

Location

Service / Stage

Instances

DMP

Service Nodes

Fast

Cache

$$$

Image Processing

Client

Where? /fast/cache/location

Learn more @ http://blog.digitalglobe.com/2016/12/01/hauling-data-boulders-from-space/

What can we do with our data freed from the dungeon?

Collect data Create job Copy data Moved to S3

Challenges at the EdgeM

IGR

AT

ION

CO

LL

EC

TIO

N

Create job Copy data Moved to S3

Snowball Edge broad strokes

Protected

and stored

CO

LL

EC

TIO

N

Accessible and

consumable

Short

time frame

Migrate to

the cloud

Snowball Edge broad strokes

Lambda function

Greengrass

integration

CloudIoT

Device

Snowball

clusters

New interfaces (HW)

and Software (S3 & NFS)

Snowball Edge Key Features

Extension of your

data center

Process data Expedites

move

Encrypted, secure,

and embedded

compute

Write data directly

as data is generated

Offers a fast and

cost effective way to

ensure data can be

quickly transferred to

and from the cloud

Simplifies

data transfer

Uses standard

and familiar tools

for the data

transfer process

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Rich Ridolfo, Philips Healthcare

Philips HealthSuite & AWS

Philips HealthSuite & AWS

• What is HealthSuite?

• What are the challenges of IT in the Clinical Healthcare realm?

• Snowball Edge in HealthSuite

30

Clinical Healthcare IT Challenges

• Reliability / Availability / Serviceability – Always Up

• Performance – 3 seconds or less [500MB – 2TB per object]

• Connectivity – Capacity and Reliability

• User Perception / Trust

Mitigation – On-site edge device

31

Technology is now key to the delivery of care

Connectivity

Lack of access to broadband [ 25Mbps down / 3Mbps up ] affects:

• 10% of all Americans ( 34 million people )

• 39% of rural Americans ( 23 million people )

• 41% of Americans living on Tribal lands ( 1.6 million people)

• 68% living in rural areas of Tribal lands ( 1.3 million people )

• 66% of Americans living on U.S. territories ( 2.6 million people )

• 98% living in rural areas territorial areas (1.1 million people )

Source: https://www.fcc.gov/reports-research/reports/broadband-progress-reports/2016-broadband-progress-report

Broadband internet access remains a challenge through the US

Desired Solution Characteristics

Survivability / Severability

Universal Codebase

End-to-end Security

Redundancy

Scalability

Single point of management

33

IntelliSpace Console Critical Care

Patient Centered ICU Dashboard

FDA Class 2 Medical Device

• Visualization of data from many sources

• Clinical Decision Support & Error Prevention.

• Dynamic, Task Specific Data Displays

• Checklists & Smart Alerts

• Collaborative Workspace – Care Team, Patient & Family.

What is it?

IntelliSpace Console Critical Care

Aggregate over 1200 ICU patient data points per day

• Vital Signs

• Labs

Process – Compute – Predict

Present

Snowball Edge

Storage + Compute

Encrypted

Tamperproof

Redundant

Philips IntelliSpace Console Critical Care

HealthSuite + AWS Snowball Edge

35

Snowball Edge – Compute

• Running IntelliSpace Console Critical Care

Execute for data transformation

• Real-time data analysis and results delivery

• Site Severability

Snowball Edge – Store

• DB Host

• Disk encrypted - security

The Solution

36

Snowball Edge

On-site hub for communication, data aggregation & pre-processing.

Pathway

s

Analytics

Getting started

Learn more about Snowmobile and Snowball Edge

https://aws.amazon.

com/snowmobile

Contact

AWS Sales

Go to the

management

console

https://aws.amazon.

com/snowballedge

Snowmobile Snowball Edge

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Storage “Office Hours”Meet the People who Build AWS Storage

Who: Lead Software Development Engineers, Architects, and Technical PMs

Where: Storage Booth Walk-up Bar

When: Exhibit hours (Tues 5-7pm, Wed & Thurs 10:30a-6:00p)

What: Architecture best practices, code reviews, feature requests

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Questions & Answers

Thank you!

Batches and Streams

Direct

Connect

Snowball,

Snowball Edge,

Snowmobile

3rd Party

Connectors

Transfer

Acceleration

Storage

GatewayKinesis Firehose

File

Amazon EFS

Block

Amazon EBS (persistent)

Object

Amazon

GlacierAmazon S3 Amazon EC2

Instance Store (ephemeral)