tools and tech for big data success
Post on 19-Oct-2014
1.305 views
DESCRIPTION
What do you need to succeed in working with Big Data? RedMonk analyst Donnie Berkholz will present quantitative research on the state of the field, covering the breadth of languages, tools, and infrastructure, to show you which choices to make today and which ones you'll need to get ready for, soon.TRANSCRIPT
Tools and tech for Big Data success
Donnie Berkholz, Ph.D.IT Industry Analyst@dberkholz
[Imagine cheesy clipart of hammers, silicon, etc.]
Tools and tech — huh?
● Languages● Infrastructure● Tooling● It's not about the what, it's about the how
Thesis: Technology adoption is increasingly bottom-up
The new kingmakers
The best DX wins
It's all about barriers to entry
Ecosystems matter
The challenges
IT pros delivering data solutions60/36 IT/biz
Talend, summer 2012
The skills gap
NVP survey of execs, summer 2012
What can we learn from book sales?
What about the data analysts?
KDNuggets, summer 2012
What about the data analysts?
KDNuggets, summer 2012
It's not all about popularity...
The real growth isn't in SQL interfaces
Developers vs. marketers
Google Trends
IT pros delivering data solutions
60/36 IT/bizTalend, summer 2012
Hadoop distributions: popularity
Google Trends: $VENDOR hadoop
Developers choose what's easiest
How to run: bare metal / DC, private cloud, public cloud
Storing Data: HDFS, Ceph, Gluster
Open Building Blocks but not Open Source: EMR on AWS
Realtime: Impala, Druid
In-memory: Redis, Memcached
Streaming: Storm, S4
Other Options: Hadoop YARN, HPCC, Cassandra, Mongo, Riak
Other Options: Spark/Shark/Mesos
Abstraction: Mortar, Continuuity, Qubole, Concurrent Lingual, etc.
Conclusion: KISS
Donnie Berkholz, Ph.D.Analyst, RedMonk@dberkholz
Disclosure: 10gen, Amazon, Basho, Cloudera, Continuuity, IBM, MapR, Microsoft, and VMware are clients.