chasing the elephant
Post on 01-Nov-2014
930 Views
Preview:
DESCRIPTION
TRANSCRIPT
@iconara
CHASING THE ELEPHANT
Theo / @iconara
chief architect at BURT
big data analytics with JRuby
RUBY
RUBY
RUBY JRUBY
JRUBY IS AWESOMEBECAUSE RUBY IS GREAT,
AND THE JVM IS GREAT
hot_bunnies, eurydice, multimeter, mikka, msgpack-jruby
HADOOP
JRUBY SUPERCOMPUTING40 TiB data, 120 EC2 cc2.8xlarge, 1920 cores, 7260 GiB RAM
JAVA ALL THE WAY DOWN
HADOOP STREAMINGWukong, Dumbo
RUBYDOOPrubydoop.org
+ =
main()
Class.forName("...")
Class.forName("...")
module WordCount class Mapper def map(key, value, context) value.to_s.downcase.split.each do |word| key = Hadoop::Io::Text.new(word) value = Hadoop::Io::IntWritable.new(1) context.write(key, value) end end endend
RUBYDOOP IS LOW LEVELI would love to see someone write something
like Scalding or Cascading on top of it
RUBYDOOPrubydoop.org
RUBYDOOPrubydoop.org
v1.1.0
KTHXBAI@iconara
github.com/iconaraarchitecturalatrocities.com
burtcorp.com
top related