from zero to hero on awseventos.uanl.mx/techday/archivos/amazonws.pdf · from zero to hero on aws...

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark

From Zero to Hero on AWSEduardo Patiño - [email protected] Trejo – [email protected]

mailto:[email protected]

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark

?

… AWS: plataforma bajodemanda de cómputo en la

nube

Amazon.com…


AWS for Research & Technical Computing

Time to ScienceAccess research

infrastructure in minutes

Low CostPay-as-you-go pricing

ElasticEasily add or remove

capacity

Globally AccessibleEasily Collaborate with

researchers around the world

SecureA collection of tools to

protect data and privacy

ScalableAccess to effectively

limitless capacity


Collaboration is easier in the cloud

More time spent computing the data than moving the data.


AWS Global Infrastructure

18 Reg ions

55AZs

121 EdgeLocations


Sample US Region

Datacenter Datacenter

Datacenter

Sample Availability Zone

Availability Zone B

Availability Zone A

Availability Zone C

Anatomy of a region


A tool for almost every job


A tool for almost every jobbecause you’ll want tools you don’t know about yet


Account Support

Support

Managed Services

Professional Services

Partner Ecosystem

Training & Certification

Solution Architects

Account Management

Security & Pricing Reports

Technical Acct. Management

Marketplace

Business Applications

DevOps Tools

Business Intelligence

Security

Networking

Database & Storage

SaaS Subscriptions

Operating Systems

Mobile

Build, Test, Monitor Apps

Push Notifications

Build, Deploy, Manage APIs

Device Testing

Identity

Enterprise Applications

Document Sharing

Email & Calendaring

Hosted Desktops

Application Streaming

Backup

Game Development

3D Game Engine

Multi-player Backends

Mgmt. Tools

Monitoring

Auditing

Service Catalog

Server Management

Configuration Tracking

Optimization

Resource Templates

Automation

Analytics

Query Large Data Sets

Elasticsearch

Business Analytics

Hadoop/Spark

Real-time Data Streaming

Orchestration Workflows

Managed Search

Managed ETL

Artificial Intelligence

Voice & Text Chatbots

Machine Learning

Text-to-Speech

Image Analysis

IoT

Rules Engine

Local Compute and Sync

Device Shadows

Device Gateway

Registry

Hybrid Devices & Edge SystemsData Integration Integrated

NetworkingResource

ManagementVMware on

AWSIdentity

Federation

Migration Application Discovery

Application Migration

Database Migration

Server MigrationData Migration

Infrastructure Regions Availability Zones

Points of Presence

Compute Containers Event-driven Computing

Virtual Machines Simple Servers Auto Scaling BatchWeb

Applications

Storage Object Storage Archive Block Storage Managed File Storage

Exabyte-scale Data Transport

Database MariaDB Data Warehousing NoSQLAurora MySQL Oracle SQL ServerPostgreSQL

ApplicationServices Transcoding Step Functions Messaging

Security Certificate Management

Web App. Firewall

Identity & Access

Key Storage & Management

DDoS Protection

Application AnalysisActive Directory

Dev Tools

Private Git Repositories

Continuous Delivery

Build, Test, and Debug

Deployment

Networking Isolated Resources

Dedicated Connections Load Balancing Scalable DNSGlobal CDN

The AWS Platform


With AWS, Security Is a Shared Responsibility

DatabaseStorageCompute Networking

Edge Locations

Regions

Avail. Zones


Customers are responsible for

security ‘in’ the Cloud

AWS is responsible for security ‘of’ the Cloud

Customer Data

Platform, Applications, Identity & Access Management

Operating System, Network & Firewall Configuration

Client-side Data Encryption & Data

Integrity Authentication

Server-side Encryption (File System and/or

Data)

Network Traffic Protection (Encryption /

Integrity / Identity)


EC2 Compute Instance Types

Well suited for AI/ML simulation and inference workloads

Well suited for AI/ML training workloads


Clusters in the cloud are fit for purpose

Amazon S3

R4P2 P2 P2

P2 P2 P2 R4C5

C5

C5 C5

C5 C5


But it’s not (just) about servers…


§Cuanto Alm

acenamiento

necesito?

§ ¿No tengo un cátalo de

Software

preaprobado?§ ¿Necesito HSM?

§¿có

mo Audito to

do lo

que pasa en mi ce

ntro

de datos?§ ¿cómo Automatizo si

se viola alguna politica?

§ ¿Cambio de configuración?

§ ¿Y si tengo un DDoS?

§ ¿Y si la versión no compila ?

§ ¿cuáles son las

herramientas

modernas que

solucionan los

problemas del pasado?

§ ¿Sera necesario tener

un DRP?

§ ¿LOS CONTRATOS

ACTUALES SON

COMPLEJOS?

§ ¿Probar Software que cumpla tarda meses?

§ ¿No tengo Especialista

para Instalar cada App?

§ Cuantos Servidores§ Prevención de perdida

de Datos !! Suena Bonito pero Costoso§ ¿Auditar Logs !! ?

Escasamente los

puede guardar 1

semana !!!

§¿y la seguirdad y la

auditoria ?§ ¿Encriptar o No

Encriptar?


Physical Virtualisation Containerization Serverless

Evolving Compute Abstractions


Big DataTechnologies and techniques for

working productively with massive amounts of data at any scale in

either batch or real-time.


Velocity Volume

Variety

Three Vs of Big Data


GB TBPB

ZB

EB

Big Data: Unconstrained Growth

Unstructured data growth is explosive

95% of the 1.2 zettabytes of data in the digital universe is unstructured

Machine data and IoT will only steepen the curve

70% of this data is user-generated content

Source: IDC, The Internet of Things: Getting Ready to Embrace Its Impact on the Digital Economy, March 2016.

http://www.idc.com/getdoc.jsp?containerId=DR2016_GS4_VT


Relational

NoSQL

Web servers

Mobile

phones/Tablets

3rd party feeds

IoT

Clickstream

Sources

Big Data Sources


Structured

Unstructured

Text

Binary

Real-time/Near Real-time

Batched

Formats

Big Data Formats and Velocity

Velocity


Managed Services for Analytics

Retrospectiveanalysis and

reporting

Here-and-nowreal-time processing

and dashboards

Predictionsto enable smart

applications


Why Big Data?

Security threat detection

User Behavior Analysis

Smart Application (Machine Learning)

Business Intelligence

Fraud detection

Financial Modeling and Forecasting

Spending optimization

Real-time alerting

Get answers faster and be able to ask questions not possible to today.


Elastic and highly scalable

No upfront capital expense

Only pay for what you use+

+

Available on-demand+

= the Cloud removes constraints


No one tool rules them all


AWS Big Data Platform

EMR EC2

Glacier

S3

Import Export

Kinesis

Direct Connect

Machine LearningRedshift

DynamoDB

AWS Database Migration Service

Collect Orchestrate Store Analyze

AWS Lambda

AWS IoT

AWS Data Pipeline

Amazon Kinesis Analytics

AmazonSNS

AWS Snowball

AmazonSWF

Amazon Athena

Amazon QuickSight

Amazon AuroraAWS Glue


Big data ≠ big cost


Sample Reference Architecture: Data Lake

AthenaGlue


Some AWS Big Data Services


• Scalable object storage for the Internet• 1 byte to 5 TB in size per object +

unlimited number of objects• 99.999999999% durability, 99.99%

availability• Regional service, no single points of

failure• Server side encryption

Storage


App Services

Deployment & Administration

Networking

Compute Database

Analytics

Amazon S3


Why Is Amazon S3 Good for Big Data?

• Natively supported by big data frameworks (Spark, Hive, Presto, etc.) • No need to run compute clusters for storage (unlike HDFS)• Must be used for transient Hadoop clusters & Amazon EC2 Spot instances• Multiple distinct (Spark, Hive, Presto) clusters can use the same data• Unlimited number of objects • Very high bandwidth – no aggregate throughput limit• Tiered-storage (Standard, IA, Amazon Glacier) via life-cycle policy• Secure – SSL, client/server-side encryption at rest• Low cost


Object Storage Use Cases

S3

S3-IA

Glacier

Cloud Applications

Big DataAnalytics

Content Distribution Primary Data

File Sync & Share

ActiveArchive

EnterpriseBackup

MediaTranscoding

Disaster Recovery /Geo Redundancy

Deep / Offline

Archives

Tape Vaulting Replacement

WORM Compliant

Data

Temporary & Small

Objects


L

i

f

e

c

y

c

l

e

AvailableS3: 99.99%

S3-IA: 99.9%

PerformantLow Latency

High Throughput≥ 30 Days≥ 128K

≥ 90 Days

Durable99.999999999%

ScalableElastic capacity No preset limits

> 0K$0.004 / GB per month

$0.0125 / GB per month

“Hot” DataActive and/or

Temporary Data

“Warm” DataInfrequently

Accessed Data

“Cold” DataArchive and

Compliance Data

≥ 0 Days> 0KStarts at $0.023 / GB per month

1-5 mins

$0.01/GB retrieval

Storage Tiered To Your Requirements

S3-IA

Glacier

S3

3 new retrieval options

3–5 hrs 5–12 hrs

Expedited Standard Bulk$0.03 / GB $0.01 / GB $0.0025 / GB


Amazon Glacier•Low-Cost Archival Storage•Secure

• SSL & AES-256

•Durable• Designed for 99.999999999% durability

•Optimized for data archiving and backup• Suitable for RTO measured in hours• Includes storage costs and retrieval costs

•Three retrieval options: Expedited, Standard, Bulk •As little as $0.004 per GB/Month (US East pricing)•Integrated with S3


Query and analyze Amazon S3 data with

standard (ANSI) SQL

No ETL required

Serverless and simple

Pay only for queries you run

Amazon Athena

Storage


App Services


Networking

Analytics

Compute Database


BI service performs ad-hoc analysis

Build intuitive visualizations

Share and collaborate via storyboards

Native access on major mobile platforms

Fast insights with SPICE

Amazon QuickSight

Storage


App Services


Networking

Analytics

Compute Database


Open Data at AWS


IT’S ABOUT SCIENCE, NOT SERVERS.

aws.amazon.com/rcp

We recognize that whilst research is often a compute-intensive activity, most researchers are not IT experts.

We want to simplify research in the cloud with easy-to-use tools for researchers and their students, and share the catalogue of “researcher-obsessed” products and services created by many of our partners.

ElevationModels

AerialImagery

ClimateModels

SatelliteImagery

High-resolutionRadar

aws.amazon.com/earth

http://aws.amazon.com/earth


Landsat 8 Lives on AWS

Within the first year of Landsat on AWS, the data has been requested over 1

billion times, globally. Over 400,000 scenes

are now available


Machine Learning on AWS


A long heritage of machine learning at Amazon

Personalized recommendations

Inventing entirely new customer experiences

Fulfillmentautomation and

inventory management

Drones Voice driven interactions


The Amazon Machine Learning Stack

FRAMEWORKS & INTERFACES

Caffe2 CNTKApache MXNet

PyTorch TensorFlow Chainer Keras Gluon

AWS Deep Learning AMIs

Amazon SageMaker

Rekognition Transcribe Translate Polly Comprehend Lex

AWS DeepLens

EDUCATION

PLATFORM SERVICES

APPLICATION SERVICES

APPLICATION SERVICES


API-driven Services

Services

Platforms

Frameworks

Infrastructure

Apache MXNet TorchCognitiv

e ToolkitKerasTheanoCaffe2 & CaffeTensorFlow

AWS Deep Learning AMI

GPU MobileCPU IoT

Amazon ML ECSSpark & EMR Kinesis Batch

Vision Speech Language


Amazon RekognitionDeep learning-based image recognition serviceSearch, verify, and organize millions of images

Object and SceneDetection

FacialAnalysis

FaceComparison

FacialRecognition

Integrated with S3, Lambda, Polly, Lex


Amazon Rekognition

Potential Use CasesSearchable Image Library Detect Inappropriate Content in

ImagesFace-based User Verification Sentiment AnalysisFacial Recognition Celebrity Identification


Amazon PollyPotential Use CasesContent Creation Education & E-learningMobile & Desktop Applications Customer Contact CenterInternet of Things (IoT) Accessibility


Amazon Lex

Potential Use CasesAppointment Booking Customer SupportInformational Services Access Enterprise DataInternet of Things (IoT)


Building better machine learning models for recommendation systems

Airbnb is using machine learning to optimize search recommendations and improve dynamic pricing guidance for hosts, both of which translate to increased booking conversions. With Amazon EC2 P3 instances, Airbnb has the ability to run training workloads faster, go through more iterations, build better machine learning models and reduce cost.


http://aws.amazon.com/free

http://aws.amazon.com/free


Como empezamos con AWS ?

1. Inscríbase en el programa AWS Educate de manera gratuita www.awseducate.com

2. Abra una cuenta gratuita de AWS y use servicios de manera gratuita https://aws.amazon.com/es/free/

3. Tutoriales de 10 minutos - https://aws.amazon.com/es/getting-started/4. Entrenamiento - https://aws.amazon.com/es/training/5. Contacte a su gerente de cuenta - Mauricio Trejo [email protected]. Involucre a un Arquitecto de Soluciones para explorar

• Infraestructura y Servicios principales• Disaster Recovery y Migraciones• Seguridad y Cumplimiento, DevOps• Big data, Inteligencia de negocios, Inteligencia Artificial• Aplicaciones Enterprise

http://www.awseducate.com/

https://aws.amazon.com/es/getting-started/


Thank you!

Eduardo PatinoSolutions Architect, AWS

from zero to hero on awseventos.uanl.mx/techday/archivos/amazonws.pdf · from zero to hero on aws...

Documents