snia abds 2015: trends and predictions for big data...overview, mostly in order • in our previous...
TRANSCRIPT
Overview, Mostly in Order • In our previous episode • The value of data • Major trends in the evolution of Big Data
services and solutions • Predictions for 2015 and beyond
– Important technologies – Spending, market growth – Issues and complications
• Integrating Big Data into your workflow
Last Year’s Talk 1. There is a real Big Data opportunity
– Money being spent – Investment in new technologies and solutions – Non-HPC enterprises will look at HPC solutions
2. Much broader than Hadoop – Many different faces to Big Data – Primarily in-house applications and algorithms
3. Scalable I/O performance most important area – Top “satisfaction gap” for all segments – Conversely, significant barriers to public cloud
Big Data: Still a Big Deal
• Big Data is bolstered by other trends – IoT: Creates more data – Cloud: Increased access to data – Mobile: Creates more data AND increases access
• Big Data is growing faster than ever, and won’t stop. The question is, What do we do with it?
“Big Data”: The rapid growth in the creation and accessibility of more data, and more types of data, stressing organizations’
ability to manage it or create competitive advantage from it.
Make Money, Of Course • Assumption: “If data, then value” • Use data for business advantage:
– Predict customer behavior – Dynamic pricing – Risk reduction
• And some altruistic stuff too: – Prevent spread of disease – Catch bad guys (more on this later) – Make the world a better place
An Analogy: ERP, Then vs. Now
Solutions Evolving for Big Data • Commercialized frameworks
– More than just Hadoop – Requires understanding of the specific problem
• Big-memory appliances for in-memory database – The return of shared memory! – Amplified by changes in processing architectures
• “Data-centric” computing – Move the computation, not the data (?) – Focus on complete workflows / data flows, not flops
Important Technologies • Parallel file systems • Windows and mobile clients • Programming environments • Middleware (job scheduling, optimization, …) • (Okay, sure …) Flash • On the other hand:
– Keep Hadoop in its appropriate place – Not so much public cloud – Sorry, there’s still no “killer app”
Growth in the “Big Data Market” • First off, what does that mean, specifically? • Certainly there are more organizations adopting
Big Data initiatives, but usually with: – Infrastructure they already have – People they already have – Maybe new software, but much of it open-source or
in-house • Most real spending is coming from planned
upgrades in normal refresh cycles • Big Data could help drive enterprise growth soon
Help Wanted • New applications and architectures create a
sudden need for new skills. From where? • Even without Big Data, new processor
architectures cause reinvention in skills for normal titles like “administrator”
• Programming skills and services will be in particularly short supply
• Warning: A bad analyst can do a lot of bad analysis with good data
The Other Issue (Time Permitting)
QUIZ: Who said it, and in what year?
“You zero privacy anyway. Get over it.”
The Other Issue (Time Permitting)
QUIZ: Who said it, and in what year?
“You zero privacy anyway. Get over it.”
Answer: Scott McNealy,
CEO, Sun Microsystems
1999
Remember This?
What is your right to privacy? Does it matter if you are famous? … in public office? … a minor?
Integrating Big Data • Performance and scalability are important
enterprise metrics now • Evaluating performance is more complex than
flops, GB/sec, or IOPS • What is the goal of your Big Data initiative?
– How will you integrate Big Data into your existing workflow?
– Where will the skills come from? – What is the ROI specifically?
