concurrency and distributed systems using jruby
DESCRIPTION
Presented at JRubyConf EU August 2012TRANSCRIPT
![Page 1: Concurrency and Distributed Systems Using JRuby](https://reader033.vdocument.in/reader033/viewer/2022051411/54579565b1af9f39378b52f2/html5/thumbnails/1.jpg)
CONCURRENCYAND DISTRIBUTED SYSTEMS IN JRUBY
@iconara
![Page 2: Concurrency and Distributed Systems Using JRuby](https://reader033.vdocument.in/reader033/viewer/2022051411/54579565b1af9f39378b52f2/html5/thumbnails/2.jpg)
speakerdeck.com/u/iconara(real time!)
![Page 3: Concurrency and Distributed Systems Using JRuby](https://reader033.vdocument.in/reader033/viewer/2022051411/54579565b1af9f39378b52f2/html5/thumbnails/3.jpg)
Theo / @iconara
![Page 4: Concurrency and Distributed Systems Using JRuby](https://reader033.vdocument.in/reader033/viewer/2022051411/54579565b1af9f39378b52f2/html5/thumbnails/4.jpg)
Chief Architect at
![Page 5: Concurrency and Distributed Systems Using JRuby](https://reader033.vdocument.in/reader033/viewer/2022051411/54579565b1af9f39378b52f2/html5/thumbnails/5.jpg)
big data with Ruby since 2009
![Page 6: Concurrency and Distributed Systems Using JRuby](https://reader033.vdocument.in/reader033/viewer/2022051411/54579565b1af9f39378b52f2/html5/thumbnails/6.jpg)
MAKING THIS
![Page 7: Concurrency and Distributed Systems Using JRuby](https://reader033.vdocument.in/reader033/viewer/2022051411/54579565b1af9f39378b52f2/html5/thumbnails/7.jpg)
INTO THIS
![Page 8: Concurrency and Distributed Systems Using JRuby](https://reader033.vdocument.in/reader033/viewer/2022051411/54579565b1af9f39378b52f2/html5/thumbnails/8.jpg)
big data with Ruby since 2009
![Page 9: Concurrency and Distributed Systems Using JRuby](https://reader033.vdocument.in/reader033/viewer/2022051411/54579565b1af9f39378b52f2/html5/thumbnails/9.jpg)
APRIL 2011stuck
![Page 10: Concurrency and Distributed Systems Using JRuby](https://reader033.vdocument.in/reader033/viewer/2022051411/54579565b1af9f39378b52f2/html5/thumbnails/10.jpg)
MAY 2011let’s try JRuby
![Page 11: Concurrency and Distributed Systems Using JRuby](https://reader033.vdocument.in/reader033/viewer/2022051411/54579565b1af9f39378b52f2/html5/thumbnails/11.jpg)
JUNE 2011wrapping Java libraries for fun and for profit
![Page 12: Concurrency and Distributed Systems Using JRuby](https://reader033.vdocument.in/reader033/viewer/2022051411/54579565b1af9f39378b52f2/html5/thumbnails/12.jpg)
HOTBUNNIESnatural Ruby API for RabbitMQ
![Page 13: Concurrency and Distributed Systems Using JRuby](https://reader033.vdocument.in/reader033/viewer/2022051411/54579565b1af9f39378b52f2/html5/thumbnails/13.jpg)
JULY 2011threads are better than processes
![Page 14: Concurrency and Distributed Systems Using JRuby](https://reader033.vdocument.in/reader033/viewer/2022051411/54579565b1af9f39378b52f2/html5/thumbnails/14.jpg)
JRUBY
![Page 15: Concurrency and Distributed Systems Using JRuby](https://reader033.vdocument.in/reader033/viewer/2022051411/54579565b1af9f39378b52f2/html5/thumbnails/15.jpg)
JRUBY
![Page 16: Concurrency and Distributed Systems Using JRuby](https://reader033.vdocument.in/reader033/viewer/2022051411/54579565b1af9f39378b52f2/html5/thumbnails/16.jpg)
J.U.C(java.util.concurrent)
![Page 17: Concurrency and Distributed Systems Using JRuby](https://reader033.vdocument.in/reader033/viewer/2022051411/54579565b1af9f39378b52f2/html5/thumbnails/17.jpg)
SHORTCUTJava Concurrency in Practice
![Page 18: Concurrency and Distributed Systems Using JRuby](https://reader033.vdocument.in/reader033/viewer/2022051411/54579565b1af9f39378b52f2/html5/thumbnails/18.jpg)
EXECUTORSconfigurable thread pools, a better
abstraction than using Thread directly
![Page 19: Concurrency and Distributed Systems Using JRuby](https://reader033.vdocument.in/reader033/viewer/2022051411/54579565b1af9f39378b52f2/html5/thumbnails/19.jpg)
thread_pool = Executors.new_fixed_thread_pool(16)
all_the_things.each do |thing| thread_pool.submit do crunch_crunch_crunch(thing) endend
![Page 20: Concurrency and Distributed Systems Using JRuby](https://reader033.vdocument.in/reader033/viewer/2022051411/54579565b1af9f39378b52f2/html5/thumbnails/20.jpg)
BLOCKING QUEUESeasy producer/consumer patterns
![Page 21: Concurrency and Distributed Systems Using JRuby](https://reader033.vdocument.in/reader033/viewer/2022051411/54579565b1af9f39378b52f2/html5/thumbnails/21.jpg)
J.U.C.ATOMICoptimistic locking primitives,
compare-and-swap, thread safe non-blocking counters
![Page 22: Concurrency and Distributed Systems Using JRuby](https://reader033.vdocument.in/reader033/viewer/2022051411/54579565b1af9f39378b52f2/html5/thumbnails/22.jpg)
MORE AWESOMESConcurrentHashMap, CountDownLatch,
ForkJoinPool, Google Guava, LMAX distruptor
![Page 23: Concurrency and Distributed Systems Using JRuby](https://reader033.vdocument.in/reader033/viewer/2022051411/54579565b1af9f39378b52f2/html5/thumbnails/23.jpg)
TL; DRuse j.u.c and avoid mutable state
and you’re safe, mostly
![Page 24: Concurrency and Distributed Systems Using JRuby](https://reader033.vdocument.in/reader033/viewer/2022051411/54579565b1af9f39378b52f2/html5/thumbnails/24.jpg)
enough ropeTHREADS
![Page 25: Concurrency and Distributed Systems Using JRuby](https://reader033.vdocument.in/reader033/viewer/2022051411/54579565b1af9f39378b52f2/html5/thumbnails/25.jpg)
AUGUST 2011threads are a mess, let’s try actors
![Page 26: Concurrency and Distributed Systems Using JRuby](https://reader033.vdocument.in/reader033/viewer/2022051411/54579565b1af9f39378b52f2/html5/thumbnails/26.jpg)
ACTORSa higher level abstraction of concurrency
![Page 27: Concurrency and Distributed Systems Using JRuby](https://reader033.vdocument.in/reader033/viewer/2022051411/54579565b1af9f39378b52f2/html5/thumbnails/27.jpg)
AKKAconcurrency library for Scala,famous actor implementation
![Page 28: Concurrency and Distributed Systems Using JRuby](https://reader033.vdocument.in/reader033/viewer/2022051411/54579565b1af9f39378b52f2/html5/thumbnails/28.jpg)
MIKKAAkka wrapper for JRuby
![Page 29: Concurrency and Distributed Systems Using JRuby](https://reader033.vdocument.in/reader033/viewer/2022051411/54579565b1af9f39378b52f2/html5/thumbnails/29.jpg)
DO ALL THE THINGSProgramming Concurrency on the JVM
![Page 30: Concurrency and Distributed Systems Using JRuby](https://reader033.vdocument.in/reader033/viewer/2022051411/54579565b1af9f39378b52f2/html5/thumbnails/30.jpg)
AUGUST 2011async systems with no back pressure
mechanism will die of memory starvation
![Page 31: Concurrency and Distributed Systems Using JRuby](https://reader033.vdocument.in/reader033/viewer/2022051411/54579565b1af9f39378b52f2/html5/thumbnails/31.jpg)
STATE
AGGREGATION
INPUT
PERSISTENCE
OUTPUT
TIMER
![Page 32: Concurrency and Distributed Systems Using JRuby](https://reader033.vdocument.in/reader033/viewer/2022051411/54579565b1af9f39378b52f2/html5/thumbnails/32.jpg)
STATE
AGGREGATION
INPUT
PERSISTENCE
OUTPUT
TIMER
!
![Page 33: Concurrency and Distributed Systems Using JRuby](https://reader033.vdocument.in/reader033/viewer/2022051411/54579565b1af9f39378b52f2/html5/thumbnails/33.jpg)
AUGUST 2011actors are cool, but not really
worth it when all the arrows in your flow chart point in the same direction
![Page 34: Concurrency and Distributed Systems Using JRuby](https://reader033.vdocument.in/reader033/viewer/2022051411/54579565b1af9f39378b52f2/html5/thumbnails/34.jpg)
DIVIDE DIVIDE DIVIDErun multiple independent, synchronous workers,
tune to the number of CPUs & IO wait
![Page 35: Concurrency and Distributed Systems Using JRuby](https://reader033.vdocument.in/reader033/viewer/2022051411/54579565b1af9f39378b52f2/html5/thumbnails/35.jpg)
SEPTEMBER 2011blocking all the way down,
back pressure all the way up
![Page 36: Concurrency and Distributed Systems Using JRuby](https://reader033.vdocument.in/reader033/viewer/2022051411/54579565b1af9f39378b52f2/html5/thumbnails/36.jpg)
AUGUST 2011(short rant about MongoDB’s global write lock)
![Page 37: Concurrency and Distributed Systems Using JRuby](https://reader033.vdocument.in/reader033/viewer/2022051411/54579565b1af9f39378b52f2/html5/thumbnails/37.jpg)
NEW SHINY TOYSfor building new shiny products
![Page 38: Concurrency and Distributed Systems Using JRuby](https://reader033.vdocument.in/reader033/viewer/2022051411/54579565b1af9f39378b52f2/html5/thumbnails/38.jpg)
CASSANDRAdistributed database, mostly magic
![Page 39: Concurrency and Distributed Systems Using JRuby](https://reader033.vdocument.in/reader033/viewer/2022051411/54579565b1af9f39378b52f2/html5/thumbnails/39.jpg)
CASSANDRAvery verbose drivers
ColumnFamilyUpdater<String, String> updater = template.createUpdater("a key");updater.setString("domain", "www.datastax.com");updater.setLong("time", System.currentTimeMillis());
try { template.update(updater);} catch (HectorException e) { // hurgh}
![Page 40: Concurrency and Distributed Systems Using JRuby](https://reader033.vdocument.in/reader033/viewer/2022051411/54579565b1af9f39378b52f2/html5/thumbnails/40.jpg)
EURYDICEJRuby nails it
new_data = {:domain => 'www.datastax.com', :time => Time.now.to_i}column_family.update('a key', new_data)
![Page 41: Concurrency and Distributed Systems Using JRuby](https://reader033.vdocument.in/reader033/viewer/2022051411/54579565b1af9f39378b52f2/html5/thumbnails/41.jpg)
STORMstream processing framework written in Clojure
![Page 42: Concurrency and Distributed Systems Using JRuby](https://reader033.vdocument.in/reader033/viewer/2022051411/54579565b1af9f39378b52f2/html5/thumbnails/42.jpg)
STORM(it does the arrows, you do the blobs)
![Page 43: Concurrency and Distributed Systems Using JRuby](https://reader033.vdocument.in/reader033/viewer/2022051411/54579565b1af9f39378b52f2/html5/thumbnails/43.jpg)
REDSTORMStorm for JRuby
![Page 44: Concurrency and Distributed Systems Using JRuby](https://reader033.vdocument.in/reader033/viewer/2022051411/54579565b1af9f39378b52f2/html5/thumbnails/44.jpg)
DON’T DO THIS AT HOME KIDS!the key to building concurrent and distributed
systems is to separate what you want to do from the concurrent- and distributed-ness, and let someone
else take care of that
YOUR CODE
![Page 45: Concurrency and Distributed Systems Using JRuby](https://reader033.vdocument.in/reader033/viewer/2022051411/54579565b1af9f39378b52f2/html5/thumbnails/45.jpg)
JRUBY IS A SLIPPERY SLOPE TO JBOSS
let there be no doubt about it
![Page 46: Concurrency and Distributed Systems Using JRuby](https://reader033.vdocument.in/reader033/viewer/2022051411/54579565b1af9f39378b52f2/html5/thumbnails/46.jpg)
SCALING IS HARDgo to Srdan’s talk tomorrow
![Page 47: Concurrency and Distributed Systems Using JRuby](https://reader033.vdocument.in/reader033/viewer/2022051411/54579565b1af9f39378b52f2/html5/thumbnails/47.jpg)
KTHXBAI@iconara
github.com/iconaraarchitecturalatrocities.com
burtcorp.com