analytics in the cloud flash talk
TRANSCRIPT
![Page 1: Analytics in the Cloud Flash Talk](https://reader034.vdocument.in/reader034/viewer/2022051522/58a1a1b71a28abac578b91c5/html5/thumbnails/1.jpg)
©"2016,"Amazon"Web"Services,"Inc."or"its"Affiliates."All"rights"reserved.
January'10,'2017
Analytics"in"the"Cloud"from"AWS
AWS GovCloud (US)
![Page 2: Analytics in the Cloud Flash Talk](https://reader034.vdocument.in/reader034/viewer/2022051522/58a1a1b71a28abac578b91c5/html5/thumbnails/2.jpg)
Global&Infrastructure
![Page 3: Analytics in the Cloud Flash Talk](https://reader034.vdocument.in/reader034/viewer/2022051522/58a1a1b71a28abac578b91c5/html5/thumbnails/3.jpg)
Global&Infrastructure&of&AWS
17* Regions – 44 Availability'Zones – 68 Edge'Locations
Region &.Number.of.Availability.Zones
AWS.GovCloud (2) EU
*C2S.(2) Ireland'(3)US West Frankfurt (2)Oregon (3) London'(2)Northern California'(3)US.East Asia.Pacific
N. Virginia'(5),'Ohio'(3) Singapore'(2)South America Sydney (2),'Tokyo'(3),São'Paulo'(3) Seoul (2),'Mumbai'(2)Canada
Central'(2) China
Beijing'(2)Announced.Regions
Paris,'Ningxia
AWS GovCloud (US)
![Page 4: Analytics in the Cloud Flash Talk](https://reader034.vdocument.in/reader034/viewer/2022051522/58a1a1b71a28abac578b91c5/html5/thumbnails/4.jpg)
AWS&GovCloud&(US)&is&an&isolated&AWS®ion
Intended&for&customers&with&strict®ulatory&and&compliance&requirements&and&sensitive&data&or&workloads
August"2011Available'to'qualified'customers
ComplianceSafeguard'sensitive'data/systems
Addresses'multiple'US'Government'regulations'and'security'requirements
![Page 5: Analytics in the Cloud Flash Talk](https://reader034.vdocument.in/reader034/viewer/2022051522/58a1a1b71a28abac578b91c5/html5/thumbnails/5.jpg)
AWS&GovCloud&(US)&distinguishing&features
Separate'IAM'(unique'credentials)
Data,'network,'and'machine'isolation'from'other'regions
Dedicated'GovCloud'Management'Console
“Community'Cloud”'with'vetted'account'holders
Managed'by'US'Persons'on'US'soil
![Page 6: Analytics in the Cloud Flash Talk](https://reader034.vdocument.in/reader034/viewer/2022051522/58a1a1b71a28abac578b91c5/html5/thumbnails/6.jpg)
Requirements&for&access&to&AWS&GovCloud&(US)
Account&holder&must&be&a&US&Person&(defined&as&a&US&citizen&or&a&Green&Card&holder)
US&entity&incorporated&to&do&business&in&the&United&States&and&is&based&on&US&soil
Can&handle&export&control&data
Learn.more:'https://aws.amazon.com/govcloud`us/getting`started/
![Page 7: Analytics in the Cloud Flash Talk](https://reader034.vdocument.in/reader034/viewer/2022051522/58a1a1b71a28abac578b91c5/html5/thumbnails/7.jpg)
AWS&GovCloud&(US)&Summary
AWS&GovCloud&(US)&is&physically&and&logically&isolatedSeparate&AZs,&Management&Console,&IAM&stack,&and&service&endpointsAWS&GovCloud&(US)&is¬&just&for&the&US&GovernmentAvailable&to&qualified&contractors&and&organizations&in®ulated&industriesRemember&the&AWS&Shared&Responsibility&ModelLeverage&services&like&VPC,&IAM,&KMS&and&CloudHSM&to&secure&sensitive&workloads&and&manage&access&to&sensitive&data
AWS&GovCloud&(US)&webpage:&https://aws.amazon.com/govcloudQus/Contact&AWS&GovCloud&team&at&[email protected]
AWS GovCloud (US)
![Page 8: Analytics in the Cloud Flash Talk](https://reader034.vdocument.in/reader034/viewer/2022051522/58a1a1b71a28abac578b91c5/html5/thumbnails/8.jpg)
C2S'brings'Amazon'Web'Services'(AWS)'“over'the'fence”'and'into'the'IC.''
C2S'is'an'“air`gapped”'region'that'utilizes'the'same'innovation'as'AWS'commercial'regions.''
AWS'will'bring'in'cutting'edge'products'and'services'for'you'to'utilize'as'technology'grows'and'changes.'And'you'only'pay'for'what'you'use.
IC'ITE'Commercial'Cloud:'AWS'C2S
![Page 9: Analytics in the Cloud Flash Talk](https://reader034.vdocument.in/reader034/viewer/2022051522/58a1a1b71a28abac578b91c5/html5/thumbnails/9.jpg)
Compute,&Analytic&and&Big&Data&Services
![Page 10: Analytics in the Cloud Flash Talk](https://reader034.vdocument.in/reader034/viewer/2022051522/58a1a1b71a28abac578b91c5/html5/thumbnails/10.jpg)
Compute:&EC2&GPU&and&FPGA&Instance&Types• G2'` optimized'for'graphics`intensive'applications.• P2'` designed'for'general`purpose'GPU'compute'applications'using'CUDA'and'OpenCL• Newly.announced• Amazon'Ec2'F1'Instances` Custom'FPGAs• Amazon'EC2'Elastic'GPUs'for'attachable'graphics'Acceleration
![Page 11: Analytics in the Cloud Flash Talk](https://reader034.vdocument.in/reader034/viewer/2022051522/58a1a1b71a28abac578b91c5/html5/thumbnails/11.jpg)
Serverless'compute'service'that'runs'your'code'in'response'to'events.
Extend'AWS'services'with'user:defined'custom'logic.
Pay'only'for'the'requests'served'and'compute'time'required': billing'in'increments'of'100'milliseconds'
AWS'Lambda
AWS'Lambda:'Serverless event'processing
![Page 12: Analytics in the Cloud Flash Talk](https://reader034.vdocument.in/reader034/viewer/2022051522/58a1a1b71a28abac578b91c5/html5/thumbnails/12.jpg)
Take&advantage&of&new&computing&models
Develop&without&worrying&about&servers&
to&manage
Setup&your&code&to&run&in&response&to&a&variety&of&triggers
Code&is&executed&on&demand&with&
continuous&scaling
Pay&only&for&the&time&used&with&subQsecond&
metering
Serverless architectures&free&you&from&managing&infrastructure
New:%Lambda@Edge lets%you%run%code%at%CloudFront edge%locations%without%provisioning%or%managing%servers.%This%allows%developers%to%deliver%a%low%latency%user%experience%for%customized%web%applications.
AWS&Lambda
![Page 13: Analytics in the Cloud Flash Talk](https://reader034.vdocument.in/reader034/viewer/2022051522/58a1a1b71a28abac578b91c5/html5/thumbnails/13.jpg)
Amazon&Kinesis:&streaming&data&made&easyServices&make&it&easy&to&capture,&deliver,&and&process&streams&on&AWS
Kinesis"Analytics"For$all$developers,$data$scientists$
Easily&analyze&data&streams&using&standard&SQL&queries
Kinesis"FirehoseFor$all$developers,$data$scientists$
Easily&load&massive&volumes&of&streaming&data&into&S3,&Amazon&Redshift,&or&Amazon&ES
Kinesis"StreamsFor$Technical$Developers
Collect&and&stream&data&for&ordered,&replayable,&realQtime&processing
![Page 14: Analytics in the Cloud Flash Talk](https://reader034.vdocument.in/reader034/viewer/2022051522/58a1a1b71a28abac578b91c5/html5/thumbnails/14.jpg)
Distributed'search'and'analytics'engine
Managed'service'using'Elasticsearch'and'Kibana
Fully'managed': zero'admin
Highly'available'and'reliable
Tightly'integrated'with'other'AWS'servicesAmazon'Elasticsearch'
Service
Amazon'Elasticsearch ServiceSemi:structured'/'unstructured'data'processing'
![Page 15: Analytics in the Cloud Flash Talk](https://reader034.vdocument.in/reader034/viewer/2022051522/58a1a1b71a28abac578b91c5/html5/thumbnails/15.jpg)
ETL&with&AWS&Data&Pipeline
![Page 16: Analytics in the Cloud Flash Talk](https://reader034.vdocument.in/reader034/viewer/2022051522/58a1a1b71a28abac578b91c5/html5/thumbnails/16.jpg)
Glue'automates'the'undifferentiated'heavy`lifting'of'ETL
! Cataloging&data&sources&
! Identifying&data&formats&and&data&types
! Generating&Extract,&Transform,&Load&code
! Executing&ETL&jobs;&managing&dependencies
! Handling&errors
! Managing&and&scaling&resources
AWS&Glue
![Page 17: Analytics in the Cloud Flash Talk](https://reader034.vdocument.in/reader034/viewer/2022051522/58a1a1b71a28abac578b91c5/html5/thumbnails/17.jpg)
AWS.Glue.Components
Data.Catalog
" Hive'metastore'compatible'metadata'repository'of'data'sources.'
" Crawls'data'source'to'infer'table,'data'type,'partition'format.'
Job.Execution
" Runs'jobs'in'Spark'containers – automatic'scaling'based'on'SLA.
" Glue'is'serverless'` only'pay'for'the'resources'you'consume.'
Job.Authoring
" Generates'Python'code'to'move'data'from'source'to'destination.'
" Edit'with'your'favorite'IDE;'share'code'snippets'using'Git.
![Page 18: Analytics in the Cloud Flash Talk](https://reader034.vdocument.in/reader034/viewer/2022051522/58a1a1b71a28abac578b91c5/html5/thumbnails/18.jpg)
Elastic&MapReduce&(EMR)• Reduces'complexity'of'big'data'processing
• Managed'Hadoop'2.7'and'2.4'infrastructure• Handles'node'provisioning,'customization,'and'shutdown• Provides'tools'to'debug'and'monitor'your'Hadoop'clusters
• Provides'tight'integration'with'AWS'services'(S3,'DynamoDB)
18
![Page 19: Analytics in the Cloud Flash Talk](https://reader034.vdocument.in/reader034/viewer/2022051522/58a1a1b71a28abac578b91c5/html5/thumbnails/19.jpg)
Amazon&Machine&Learning
• Easy`to`use'service'built'for'developers• Robust,'powerful,'and'technology`based• Ability'to'create'models'using'your'data• Deployable'to'production'in'seconds
![Page 20: Analytics in the Cloud Flash Talk](https://reader034.vdocument.in/reader034/viewer/2022051522/58a1a1b71a28abac578b91c5/html5/thumbnails/20.jpg)
Fast,'powered'by'SPICE,'automatically'scales.'
Explore,'analyze,'share'insights'with'anyone.
1/10th'the'cost'of'traditional'BI'solutions.
Broad'connectivity'with'AWS'data'services,'on:premises'data,'files'and'business'applications.
Amazon'QuickSight
Amazon'QuickSight
Amazon"RDS
Amazon"S3 Amazon"Redshift
Visualize)and)explore
![Page 21: Analytics in the Cloud Flash Talk](https://reader034.vdocument.in/reader034/viewer/2022051522/58a1a1b71a28abac578b91c5/html5/thumbnails/21.jpg)
Amazon&Athena:&Serverless Query&Processing&with&S3
• Serverless'query'service'for'querying'data'in'S3'using'standard'SQL'with'no'infrastructure'to'manage
• No'data'loading'required;'query'directly'from'Amazon'S3
• Use'standard'ANSI'SQL'queries'with'support'for'joins,'JSON,'and'window'functions
• Support'for'multiple'data'formats'include'text,'CSV,'TSV,'JSON,'Avro,'ORC,'Parquet
• Pay'per'query'only'when'you’re'running'queries'based'on'data'scanned.'If'you'compress'your'data,'you'pay'less'and'your'queries'run'faster
AmazonAthena
![Page 22: Analytics in the Cloud Flash Talk](https://reader034.vdocument.in/reader034/viewer/2022051522/58a1a1b71a28abac578b91c5/html5/thumbnails/22.jpg)
Other&re:Invent 2016&Announcements• Amazon'Lex` build'conversational'interfaces'using'voice'and'text• Amazon'Polly` turn'text'into'lifelike'speech• Amazon'Rekognition` add'image'analysis'to'your'applications• AWS'Batch` Enables'users'to'easily'and'efficiently'run'hundreds'of'thousands'of'batch'computing'jobs'on'AWS• Amazon'Lightsail` deploy'a'virtual'private'server'using'preconfigured'templates• AWS'Greengrass` run'compute,'messaging'&'data'caching'for'connected'devices'in'a'secure'way,'even'when'not'connected'to'the'Internet.• AWS'Snowball.Edge` 100TB'data'transfer'device'with'on`board'storage'and'compute• AWS'Snowmobile` Migrate'data'securely'via'a'truck'storing'up'to'100'PB'of'data
![Page 23: Analytics in the Cloud Flash Talk](https://reader034.vdocument.in/reader034/viewer/2022051522/58a1a1b71a28abac578b91c5/html5/thumbnails/23.jpg)
Implementing'a'Data'Lake'on'AWS
Elasticsearch
![Page 24: Analytics in the Cloud Flash Talk](https://reader034.vdocument.in/reader034/viewer/2022051522/58a1a1b71a28abac578b91c5/html5/thumbnails/24.jpg)
AWS&and&IC&Marketplace
24
• 3,500+'software'listings• Over'1,100'participating'ISVs
• Open'source'and'commercial'software''
• Try'before'you'buy,'experiment'cheaply
• Procure'new'or'Bring`your`own`license
• Deployed'in'Commercial'and'C2S'Regions
http://aws.amazon.com/marketplace
![Page 25: Analytics in the Cloud Flash Talk](https://reader034.vdocument.in/reader034/viewer/2022051522/58a1a1b71a28abac578b91c5/html5/thumbnails/25.jpg)
Networking AnalyticsCompute
Storage&&&Content&Delivery
Developer&Tools Management&Tools Security&&&Identity
Application&Services
Mobile&Services Database
S3 EBS Storage"Gateway Glacier EFS API"Gateway AppStream CloudSearch Elastic"
Transcoder SES SQS SWF
Device"Farm Mobile"AnalyticsCognito
SNS
RDS DynamoDB ElastiCache RedShift WorkSpaces WorkDocs WorkMail
Lambda EC2"Container"Service
Elastic"BeanstalkEC2 VPC ELB Direct"
Connect EMR Kinesis Data"Pipeline Machine"LearningRoute"53 QuickSight Elasticsearch
Service
CodeCommit CodeDeploy CodePipeline CloudWatch CloudFormation CloudTrail Config OpsWorks Service"
CatalogIdentity"&"
Access"MgmtDirectory"Service
Trusted"Advisor Cloud"HSM Key"
Mgmt"ServiceWeb"App"Firewall
Snowball
Simple"DB DB"Migration"Service
IOTIoT
HubsMobile"Hub
AWS"Mgmt"Console
Command"Line"Interface
Interaction
Libraries&and&SDKsJava Javascript Python
(boto) PHP .NET Ruby Node.js iOS Android Toolkit"for"VS
Toolkit"for"Eclipse
Tools"for"Powershell AWS"CLI
CloudFront
Enterprise&Applications
AWS'Commercial'Platform'Services
Marketplace
AWS"Marketplace
![Page 26: Analytics in the Cloud Flash Talk](https://reader034.vdocument.in/reader034/viewer/2022051522/58a1a1b71a28abac578b91c5/html5/thumbnails/26.jpg)
AWS&GovCloud&(US)&Services
Kinesis'Streams
![Page 27: Analytics in the Cloud Flash Talk](https://reader034.vdocument.in/reader034/viewer/2022051522/58a1a1b71a28abac578b91c5/html5/thumbnails/27.jpg)
Requesting&New&C2S&Services• C2S&page&on&IC&ITE&home&page:
• C2S&Roadmap&• Service&launch&announcements
• New&Service&Requests:• On&the&right&side&of&the&C2S&page,&click&“Request&New&Service”• Fill&out&the&form&with&POC&info,&list&of&new&services&or&features&
requested,&estimated&users&and&usage,&and&potential&useQcase
27
![Page 28: Analytics in the Cloud Flash Talk](https://reader034.vdocument.in/reader034/viewer/2022051522/58a1a1b71a28abac578b91c5/html5/thumbnails/28.jpg)
Migrating&existing&applications&as&quickly&as&possible
Deploying&new&applications&to&the&cloud&by&default
Cloud"computing"has"become"the"new"normal
Public&Sector&and&especially&the&US&IC&is&moving&to&AWS.
AWS&services&more&than&a&million&active&customers&every&month&in&190&countries.
Through&customer&obsession&and&an&increasing&pace&of&innovation,&AWS&is&the&fastest&technology&vendor&in&history&to&reach&a&$10B&run&rate.