Big Data Prototyping in AWS CloudSamuel YeeAWS Certified Solutions Architect

Minimum Viable Product (MVP)

Why Cloud Computing?• Rapid prototyping for Minimum Viable Product (MVP)

• No need to worry about what-to-buy, what-if-buy-wrong etc.• Scale up and out if demands pick up• Tear down at minimal cost if no demand or project failure

• Deploy anytime and everywhere• Worldwide infrastructure• Flexible, elastic, agile

• No high upfront costs• Pay-as-you-use model• Can start instantly without much cost justifications

Typical Big Data Use Cases

RetrospectiveAnalysis and reporting

Here-and-NowReal-time processing

and dashboards

PredictionsMachine learning and

smart applications

Typical Big Data Pipeline

Ingest

Process

Store

Analyse (BI)

Data Answer

Word Count Example

Ingest

Process

Store

Analyse (BI)

Data Answer

Which one to buy?

Demo• Spin your new Hadoop cluster in AWS within 15 minutes• Terminate your Hadoop cluster instantly when you don’t need it

Data Types Ingest Process Store Analyse (BI)

Flat Files

Database Data

StreamingData

Hadoop-as-a-Service

File Objects

Database Objects

NoSQL Data Store

Desktop Apps

Enterprise Apps

Open Source

What’s Next?• Measure the outcomes of your data projects

• No need – Change yourself or educate your users/bosses• Fail – Try to understand the needs of your users/bosses better. Deliver faster!• Success – Congratulation!

• Should I continue in AWS cloud?• Do I need it 24x7, anywhere?• Can I move my system and network admins to host my Big Data apps?• Do I have the resource, time & expertise to deal with the intricate details of

Hadoop cluster configurations and troubleshooting?• How do I secure my big data infrastructure?

• If answer is still no, go back to the Big Data Landscape chart. But at least you can keep your users/bosses happy and engaged with your MVP in cloud.