hpe keynote hadoop summit san jose 2016

Enterprise-grade Big DataChris EidlerVP, Solutions R&D, HPE

Inflection Point - Data Management goes Open Source

Infrastructure Layer

Data Layer

Analytics Layer

Apps Layer (Solutions)

Disruption ImpactWorkload Optimization

Building Blocks for Workload-Optimized Big Data

HPE Confidential - for HPE and Channel Partners only 3

Active Archive

• Multi-temperate storage with data governance and federated queries

• Denser TB/rack u, lower $/TB for long term storage

Data Lakes• Ingestion of multiple types / sources

of data• Batch, Interactive, Real-time

workloads• Different infrastructure requirements

Data Warehouse Modernization

• Data Staging & landing zone• Bach processing• Traditional and rack density

optimized form factors

Use Cases:

ProLiant DL300 series

Apollo 4530

Apollo 4200

Traditional 1U/2U design• Building block for

traditional Hadoop workloads

Density optimized platform block for traditional Hadoop workloads• Same spindle/core ratios

Storage optimized block• Foundation for Data lakes• Double the storage

density of traditional platform

Apollo 4510Densest Storage block• Online Archival• Object storage

A Big Data Journey…

ETL Offload Archival

Deep Learning

Event Processing

In Memory Analytics

HP Big Data Reference ArchitectureElastic Platform for Analytics

Event ProcessingLow Latency Compute

Moonshot m710x

In Memory AnalyticsBig Memory Compute

Apollo xl170r w 512G memory

Archival StorageApollo 4200 w 6TB HDD

ETL Offload High Latency Compute

Apollo xl170 w 256G memory

Deep LearningHPC Compute

Apollo xl190r w GPUs

HDFS StorageData Lake

Apollow 4200 w 3TB HDD

What If….Opportunities for Platform Optimization

The Coming Landscape

– Non-Volatile Memory– More than fast – byte addressable and persistent

– Photonics– Optical Networking will make most NVM equidistant

– Some Implications on Big Data– 90% of a database write transaction is eliminated– A Shuffle …isn’t

– HPE is contributing changes to Spark with HDP

– Favored Algorithms might change– Graph and matrix inversion based algorithms

Confidential

HPE’s “The Machine”A shared something architecture

Platform Investigations for Workload Optimized Big Data

Confidential

Silicon Acceleration

Big Data/HPC/Cloud integration

HPC Big Data Cloud

Composed Big Data

Multicore x86 CPU

GPGPU FPGA SoC/ASIC

Software Hardware

Meaning Aware Storage

Push work into storage

HPE’s Own HDP Deployment – Modernizing Data ArchitectureMillions in Savings and Significantly Improving Analytics

Data Lake Core

EA Dashboards & Reporting- Dedicated satellite- Marketplace interface- Certified reports/data- Enterprise consumption platforms

Satellite Analytics Clusters- Super user + enterprise data- Provisioned via project interlock- Services analytics tools- Domain (BU) zones and refineries (ad-hoc jobs)- Synchronized via Hadoop replication

Data Lake Core- Hadoop nucleus - Enterprise refinery- Certified enterprise data- No direct consumption for general

users- Full dataset discovery via limited

YARN containers

Foundation for HPE’s Go-Forward Data Strategy• Democratizing Analytics• Open up analytics

innovation through self service consumption and governance

• Single E2E connected Data Platform

• Serve up enterprise data w/ unprecedented speed, accuracy, simplicity and flexibility

HPE and Hortonworks Team Up

• Alliance partner for 2+ years

• HPE invested $50M in Hortonworks

• HPE CTO/EVP Martin Fink is on the Board of Hortonworks

• Close collaboration from Engineering to GTM

• Technical Collaboration• YARN Node Labels (jira YARN-796)• Spark Optimized Shuffle for big memory• LLAP performance validation

– Together we’re driving Hadoop Forward

• More Open

• More Secure

• Optimized for Performance

Many of the world’s largest enterprises put their trust in the HPE-Hortonworks team!

Learn More Here at Hadoop Summit!

A New “Sparketecture” for Modernizing your Data Warehouse

Wednesday, 11:30AM, Room 210C

Demos @ Booth 501

Play the Hadoop Trivia Game and Win! – HPE Booth

Thank you

Catch HPE Session at 11:30am Wed, Room 210C

Visit the HPE booth, complete a quiz & win a prize

12