going from big data to big answers - files.meetup.comfiles.meetup.com/10136492/intel big data use...

Intel Information Technology

Going From Big Data to Big AnswersApril 17, 2014

Ajay Chandramouly

@ajayc47


Agenda

• Impact and Value of Big Data

• Intel IT Use Cases – Bringing Value of Big Data to Intel

• Call to Action – Bringing Value of Big Data to Your Business

2


Big

Data

TEXT


Big Data AnalyticsValue

= the “Asset” = the “Action”


The Four Pillars Of Big Data

5

Volume

Massive scale and growth of unstructured data

• 80%~90% of total data

• Growing 10x~50x faster than structured (relational) data

• 10x~100x of traditional data warehousing

Velocity

Real-time rather than batch-style analysis

• Data streamed in, tortured, and discarded

• Making impact on the spot rather than after-the-fact

Variety

Heterogeneity and variable nature of Big Data

• Many different forms (text, document, image, video, ...)

• No schema or weak schema

• Inconsistent syntax and semantics

Variability

Predictive analytics for future trends and patterns

• Deep, complex analysis (machine learning, statistic modeling, graph algorithms, …), versus

• Traditional business intelligence (querying, reporting, …)

Big Data augments traditional Business Intelligence

Intel Information Technology Intel Information Technology

BIG DATAMACHINE

GENERATED

HUMANGENERATED

BUSINESSGENERATED

Edge

Scale Up

Distributed

REQUIRES DIFFERENT APPROACHES

Scale Out

NETWO

RK

STORAGECOMPUTE

Intel® Optimized Big Data

In Memory

XDW

MPP

One Size Doesn’t Fit All

LOB

IOT


The Fusion of Edge Devices and Big Data Analytics

Planogram monitoring(Real-time stock level)

Interactive display(Behavioral marketing)

Real-time transaction

Store heat map (hot merchandise, browsing history, conversion rate)

RFID

RFID

Dynamic pricingReal-time, personalized AdAuto promotion/couponSocial network connections

Surveillance camera (store statistics)

Intel Information Technology 8

Going from Data to Actionable Insight


Intel IT – What We’re Doing in Big Data

9


6,500 IT Employees59 IT sites globally

150,000 Connected Systems40,000 Handheld Devices

100,000 Intel Employees164 Intel Sites across 63 Countries

68 Data Centers25% reduction with virtualization

Inspire employees

IT is business

Changing traditional thinking

Service reliability

Intel Confidential

10

Intel Information Technology Intel Information Technology 11

Our IT Environment - Continued


IT Leadership

12

Transform

Contribute Value

Deliver Services

“License to Decide”

Strategic Relationship

“Right to Influence”

Collaborative Relationship

“Reason to Exist”

Transactional Relationship

Advanced Analytics Enables IT to Transform the Business


Intel IT Vision for Big Data Analytics

13

Priority

We run big data analytics programs in each of our key lines of businesses. Also, all our key strategic initiatives have a big data component.

Strategy

Implement an internal, cost-effective big data platform

and in- parallel build the necessary skill set within

the organization,

Approach

Gradually build business value through advanced

analytics of big data.

Business Value

The value of our big data efforts was about USD

$100M in 2013. We expect that figure to grow in 2014.

IT formed an enterprise Big Data Analytics organization which solves High Value problems


Big Data Path to Competitive Advantage

14

SalesWeb usage data

for

Marketing/Camp

aign predictions

(What)

New Biz ITIT Incident

Predictability

Context Aware

Analytics for

LBS

SecurityNetwork

Intrusion

Prediction and

Prevention

Big Data Use Cases

Tailor-made and Unique Big Data environment based on Intel needs

2011 - 12

•Defined strategy and implementation plan

•Hadoop Path-finding

•Deployed chosen MPP platform

•Acquired big data skills

•Deployed 3 big data projects (3 done)

•Completed big data distribution evaluation

•Landed internal Hadoop cluster in Prod

• Implemented Internal Hadoop Production cluster

2013

• Implement Internal Hadoop Pre-Production cluster

•Deliver a solid platform for the first set of use cases.

•Deploy internal 5+6=10 projects on top of the BI big data platform

•Deploy the qualified Big Data business use cases

•Deliver business value with this platform through the use of it.

•Expand Big Data Platforms to support use case demand.

• Setup BDP as a service with integration of IT processes.

•Prescriptive guidance for development and architecture.

•Standardize processes & tools

2014 • Expand IBD platform for the next set of

use cases. Deliver business value through the use of it.

Deploy internal 5+6=100 projects on top of the BI big data platform

• Evolve the IBD platform towards the next generation Hadoop ecosystem

Adopt IDH3 with Hadoop 2.0/YARN

Hbase for storage intensive use cases

Explore SQL on Hadoop use cases

Expand Big Data Platforms to support Enterprise BI use cases.

•Continuous improvement and expansion of platform, capabilities, guidance, process and tools.

MfgAsses feasibility

of Hadoop for

MIDAS as lower

cost solution

HR - POCTalent

Intelligence


Intel IT Multi Data Warehouse Strategy

15

Big Data is a Part of a Comprehensive BI Strategy


Intel IT’s Big Data Platform

MPP Platform

3rd-party solution

100x faster than traditional systems

Intel® Xeon® processor E7 family blades scale easily

Intel Data Platform

Based on Apache Hadoop

Optimized for Intel® Xeon processors,

SSD and 10GbE

HBase NoSql DB

Spark (In-Memory Analytics)

MPP – Massively Parallel Processing

Predictive Analytics Engine In house development

Enables real time, on-going Predictive service

Intel® Xeon® processor E7 family

Intel Data Platform


Hadoop Use CasesContextual Recommendation Engine: Provides

recommendation engine and analytic capabilities to

acquisition.

Value:

•Provides new, intelligent capabilities and map

management technologies which can be offered as paid

services

Incident Predictability: Reduces incidents, impact on users and IT

Value:

•Reduces number of new incidents and employee

support costs while boosting productivity

Web Data Mining & Customer Insight: Provides customer and network usage analytics for

Intel.com and customer advertising

Value:

•Provides means to predict and adjust product position

or pricing based on response to marketing campaigns


Contextual Recommendation Engine

Telmap Server

Offer Manager

Request for Navigation

Request for Available offers

Online –Model Inquiry

Offline -Model

Creation

Context Aware Recommendation System

Request for Offers Ranking based on User Preferences

Set of Offers for Display

Personal, context aware location based recommendations system to enrich user experience

Embed into Telmap navigation application

Provide commercial recommendations to users based on uncovered preferences

Recommendations are time, location and context aware based

Big Data Solution, Generic recommendation engine

5/5/2014


Mobile Application Screen Shot

19


Events Incidents

Symptom

table

More than 80% of event data is grouped

using only 10 event IDs .

Selection of critical fields by event

ID to facilitate Problem Managers

look into data

This is a way to look

into events we didn’t

have before… [Intel IT Problem

Manager]

Symptoms are based on group

of events.

- Volume- Frequency/day- Machine Importance- Application

Importance- Symptom effect- N. Incidents

Machine crying for

help (VOM)

Humans crying for

help (VOC)vs

Important events

bubble up with

higher Impact

Linear regression, Time Series Analysis and Predict

future Impact

Symptom Impact will

allow PMs to focus on

critical events requiring

help.

Better understanding of potential future problems since

Events data ALWAYS arrive first than Incident data.

Linear regression to predict future behavior

Event Data Incident Data

Using VOM (Voice of Machines) THEN will

help to PREDICT now VOC (Voice of

Customers)

- Tokenization- Stop-words- Steaming- Synonyms

Incident Predictability Big Picture


Customer Insight: Web AnalyticsPurpose of PoC• Prove landing and ingesting of Web Data in Hadoop.

• Determining & landing of processed data in right container

• Get learnings for Path-to-production in terms of architecture options and nuances associated with Hadoop.

Business Value• Customer Insight owns delivery of the “Profile 330” and Analytics to support SMG• ROI: $20M Smart Analytics; $10M Demand Generation

5/5/2014


Call to Action

22


23

People & Skills

CxO Program Manager Project Manager Solutions Architect Data Architect Data Engineer SE/Developer DBA


Intel’s Datacenter Portfolio

24

• Intel® Atom® & Xeon® storage solutions•SSD’s with Cache Acceleration•Luster•Next Gen NVM

•Open Network Platforms•Wind River OS•Quick Assist Technology•Silicon Photonics•Ethernet Switch & NIC Si

• Intel® Xeon® Product Family E3-E5-E7• Intel® Atom™• Intel® Xeon Phi™• Integrated graphics•TXT•AESNI

Storage Network Compute Optimized, Scalable,

Efficient Architecture

Platform & Architectural Leadership

Broadest Enabled Ecosystem

Breadth of solution for any class of Big Data Analytics platform


Res

ponsi

veE

ne

rgy

Eff

icie

nt

Hig

hA

vaila

bilit

yS

ecu

re

Intel’s Foundational Technologies Offer Advanced Solutions for Big data Analytics

Ch

oic

e

Intel’s Big Data Building Blocks

Intelligent Storage1

Scale-out Storage1

Scale-up Storage1

Intel® SSD 710 series, DC S3700 (SATA)

Intel® SSD 910 series (PCIe)

Intel® Ethernet Controllers

Intel® Ethernet Adapters

Intel® Ethernet Switch Silicon

Intel® True Scale Fabric

Compute Network Storage

Intel® Data Platform

Intel® Expressway Service Gateway

Intel® Cache Acceleration Software

Intel’s Lustre

Intel® VT and Intel® TXT

Intel® AES-NI

Software & Technologies

Intel® Xeon® Product Family E3-E5-E7

Intel® Atom™

Intel® Xeon PhiTM

Intel® Quark™

Xeon-based storage systems are available in a wide range of configuration options from the industry’s leading storage vendors

25


• Build a cost-effective, versatile Big Data platform. One Size does not fit all.

• Technology is important, but skill sets are essential.

• Ecosystem is maturing. Easier than ever to get started.

Summary

Big data analytics has led to big value across every sector


IT @ Intel: Sharing Intel IT Best Practices with the World

27

Learn more about Intel IT’s initiatives at www.intel.com/IT

Or @ajayc47

CIO and IT Perspective

IT White Papers, Audio-Video Blogs

IT-to-IT Community

http://www.intel.com/content/www/us/en/mobile-computing/intel-it-mobile-computing-enabling-global-collaboration-paper.html

http://www.intel.com/content/www/us/en/mobile-computing/intel-it-mobile-computing-enabling-global-collaboration-paper.html

going from big data to big answers - files.meetup.comfiles.meetup.com/10136492/intel big data use...

Documents