terracotta hadoop & in-memory webcast
DESCRIPTION
Hadoop is sparking a Big Data analytics revolution. But all the Hadoop insights in the world are worth nothing unless they lead to new, profitable action. To translate Hadoop insights into action in real time, more and more enterprises are combining Hadoop with the power of in-memory computing. Join us as we outline the tremendous benefits of merging Hadoop with in-memory data management, the challenges of doing so, and tips for getting started.TRANSCRIPT
© 2013 Terracotta Inc. | Internal Use Only
In-Memory & Hadoop:Real-time
Big Data Intelligence
© 2013 Terracotta Inc. 2
Your speaker
Manish DevganDirector of Product
ManagementTerracotta
© 2013 Terracotta Inc. 3
What we’ll cover in this webcast
• What’s Hadoop? (quick intro)
• Hadoop’s weaknesses
• Emerging best practices for combining Hadoop and in-memory data management
• Real-time intelligence example
• Getting started with in-memory and Hadoop
• Q & A
© 2013 Terracotta Inc. 4
4© 2013 Terracotta Inc. | Internal Use Only
What is Hadoop?
© 2013 Terracotta Inc. 5
What is ?
• Hadoop is open-source software data management framework used to draw insights from data
Components Benefits
HDFS*: Scalable & distributed Storage• Data distributed across cluster nodes• Name node keeps track of location
MapReduce: Parallel Processing of data• Splits a task for processing based on
data locality and then assembles results
• Comprises of Map() procedure for filtering & sorting and Reduce() procedure for summarizing
Scalable• Efficiently store and process large data
sets
Reliable• Get redundant storage, with failover
across cluster
Rich & Flexible• Complimentary set of tools & frameworks• Store data in any format
Economical• Deploy on commodity hardware
*Hadoop Distributed File System
© 2013 Terracotta Inc. 6
What is ?
• With Hadoop, you can ask interesting questions about your data and get answers economically
Questions Hadoop can help answer
How can I target promotions to my customers for better sales?
How risky are each of my customers?
Which advertisement should I show to optimize return?
How relevant is a result for a given search?
When will my machinery likely have a malfunction?
© 2013 Terracotta Inc. 7
7© 2013 Terracotta Inc. | Internal Use Only
Hadoop’s Weaknesses
© 2013 Terracotta Inc. 8
Hadoop’s Weaknesses
• No support for real-time insights• No support to facilitate interactive and exploratory data analysis• Challenging framework for computation beyond Map Reduce• Lacks tools for business analysts
© 2013 Terracotta Inc. 9
9© 2013 Terracotta Inc. | Internal Use Only
Emerging best practices for combining Hadoop and
in-memory data management
© 2013 Terracotta Inc. 10
Combining Hadoop and In-memory Data Management
- Businesses are looking for ways to mine real-time insights to provide competitive advantages
- Increased adoption of transactional system data for analytics is blurring the line between OLTP and OLAP
- New frameworks and products are bringing in-memory technologies to the Hadoop ecosystem
© 2013 Terracotta Inc. 11
Real-time Data Integration with Hadoop
WebApps
Mobile Apps
Dashboards & Mashups
In-memory Data Management Platform
Real-time Data Apps
Transactional Apps
Operational Intelligence
Log Data POS Data Social Media Sensors
Data Sources
EventsImages/Videos
Data Feeds
Real-timedata
Real-timeInsights
© 2013 Terracotta Inc. 12
12© 2013 Terracotta Inc. | Internal Use Only
Real-time intelligence example
© 2013 Terracotta Inc. 13
BigMemory & Hadoop in financial servicesBefore: Custom ETL connector pushing batch data
Hadoop Cluster
Big
Mem
ory
Sto
re
Short Term Transaction
Data
Long Term Transaction
Data
Rules & Triggers
Tagged Accounts
Credit Reference
Data
HDFS to BigMemory Processing
Hadoop M/R
© 2013 Terracotta Inc. 14
BigMemory & Hadoop in financial servicesToday: Streaming Data insights
Hadoop Cluster
Insights Hadoop M/R
BigMemory- Hadoop
Connector
Big
Mem
ory
Sto
re
Short Term Transaction
Data
Long Term Transaction
Data
Rules & Triggers
Tagged Accounts
Credit Reference
Data
© 2013 Terracotta Inc. 15
15© 2013 Terracotta Inc. | Internal Use Only
Getting started with in-memory and Hadoop
© 2013 Terracotta Inc. 16
How to get started with In-memory and Hadoop?
• If you already have a Hadoop project, look for use cases where you want real-time access to insights
• Start with a small-to-medium sized (20-40 nodes) cluster with a well-defined use case requiring fast access to data
• Consider exploratory use cases where you’re doing iterative analysis on a data set to get answers faster
© 2013 Terracotta Inc. 17
In-Memory & Hadoop
QuestionsPlease type yours in the “Questions” panel or in the chat window.
© 2013 Terracotta Inc. 18
Connect with Terracotta
• Download “BigMemory & Hadoop” white paper− Visit: www.terracotta.org (Resources > White Papers)
• Download “BigMemory-Hadoop Connector”− Visit: www.terracotta.org/downloads/hadoop-connector
• Contact Manish Devgan− Email: [email protected]
• Follow us on Twitter− @big_memory
• Stay Tuned