![Page 1: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/1.jpg)
Introduction to Clojure Concurrency
(and data structures)
Karl Krukow,Engineer at Trifork &
CTO LessPainful@karlkrukow
Goto Amsterdam, May 2012
fredag den 25. maj 12
![Page 2: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/2.jpg)
Introduction to Clojure Concurrency
(and data structures)
Karl Krukow,Engineer at Trifork &
CTO LessPainful@karlkrukow
Goto Amsterdam, May 2012
fredag den 25. maj 12
![Page 3: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/3.jpg)
Intro to Clojure data structures(with some parallelism added too)
Karl Krukow,Engineer at Trifork &
CTO LessPainful@karlkrukow
Goto Amsterdam, May 2012
fredag den 25. maj 12
![Page 4: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/4.jpg)
About me
• PhD Computer Science, Uni. Aarhus, 2006
• Engineer at Trifork for about 6 years on Web, JavaScript, Java/JEE, Ruby/Rails, iOS, Conferences and Training.
• Clojure and Rich Hickey fan-boy! (and Keynote newbie)
• Recently CTO of LessPainful, automated mobile app testing: (http://www.lesspainful.com).
• Conj Labs - 3 day Clojure training in Europe (with Lau Jensen)
fredag den 25. maj 12
![Page 5: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/5.jpg)
About me
• PhD Computer Science, Uni. Aarhus, 2006
• Engineer at Trifork for about 6 years on Web, JavaScript, Java/JEE, Ruby/Rails, iOS, Conferences and Training.
• Clojure and Rich Hickey fan-boy! (and Keynote newbie)
• Recently CTO of LessPainful, automated mobile app testing: (http://www.lesspainful.com).
• Conj Labs - 3 day Clojure training in Europe (with Lau Jensen)
fredag den 25. maj 12
![Page 6: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/6.jpg)
Why give a talk about Data Structures?
fredag den 25. maj 12
![Page 7: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/7.jpg)
Pop-quiz
fredag den 25. maj 12
![Page 8: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/8.jpg)
Pop-quiz//m is a shared java.util.HashMap
public static void write(final int offset) { for (int i = 0; i < 10000; i++) { int k = offset+i; m.put(k, -k); } }
public static void read(final int offset) { for (int i = 0; i < 10000; i++) { int key = offset+i; Integer val = m.get(key); if (val != null) { if (val.intValue() != -key) { System.out.println("Key and value don't match..."); } } } }
fredag den 25. maj 12
![Page 9: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/9.jpg)
Pop-quiz
Suppose we have multiple threads calling both read and write without synchronization.
Now, what can happen?
//m is a shared java.util.HashMap
public static void write(final int offset) { for (int i = 0; i < 10000; i++) { int k = offset+i; m.put(k, -k); } }
public static void read(final int offset) { for (int i = 0; i < 10000; i++) { int key = offset+i; Integer val = m.get(key); if (val != null) { if (val.intValue() != -key) { System.out.println("Key and value don't match..."); } } } }
fredag den 25. maj 12
![Page 10: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/10.jpg)
Non-obvious bug 1• ArrayIndexOutOfBoundsException?
krukow:~/workspaces/trifork/concurrency$ java -cp bin hashmap.HashMapDemoException in thread "Thread-0" java.lang.ArrayIndexOutOfBoundsException: 23 at java.util.HashMap.get(HashMap.java:301) at hashmap.HashMapDemo.read(HashMapDemo.java:17) at hashmap.HashMapDemo$1.run(HashMapDemo.java:32) at java.lang.Thread.run(Thread.java:637)WRITE done: j = 1READ done: j = 2READ done: j = 4READ done: j = 6WRITE done: j = 3READ done: j = 8READ done: j = 10
fredag den 25. maj 12
![Page 11: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/11.jpg)
Non-obvious bug 2• Infinite loop!
krukow:~/workspaces/trifork/concurrency$ java -cp bin hashmap.HashMapDemoREAD done: j = 0READ done: j = 2WRITE done: j = 1WRITE done: j = 3…READ done: j = 12WRITE done: j = 11READ done: j = 14WRITE done: j = 15WRITE done: j = 17READ done: j = 18READ done: j = 16WRITE done: j = 19^[[A^[[B^[[A^[[A^C
fredag den 25. maj 12
![Page 12: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/12.jpg)
Is this in theory only?
fredag den 25. maj 12
![Page 13: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/13.jpg)
Is this in theory only?• Of course not!
• I've seen the “infinite loop” issue put down a cluster of production servers!
fredag den 25. maj 12
![Page 14: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/14.jpg)
Is this in theory only?• Of course not!
• I've seen the “infinite loop” issue put down a cluster of production servers!
• Missing technical knowledge (e.g. JMM)
fredag den 25. maj 12
![Page 15: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/15.jpg)
Is this in theory only?• Of course not!
• I've seen the “infinite loop” issue put down a cluster of production servers!
• Missing technical knowledge (e.g. JMM)
• Incorrect optimizations:
• “I really can't pay the cost of synchronization (even though I haven't measured it), and in this particular case a data-race is safe.”
fredag den 25. maj 12
![Page 16: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/16.jpg)
Is this in theory only?• Of course not!
• I've seen the “infinite loop” issue put down a cluster of production servers!
• Missing technical knowledge (e.g. JMM)
• Incorrect optimizations:
• “I really can't pay the cost of synchronization (even though I haven't measured it), and in this particular case a data-race is safe.”
• Non-obvious sharing:
• “I thought this object wasn't shared between multiple threads.”
fredag den 25. maj 12
![Page 17: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/17.jpg)
Is this in theory only?• Of course not!
• I've seen the “infinite loop” issue put down a cluster of production servers!
• Missing technical knowledge (e.g. JMM)
• Incorrect optimizations:
• “I really can't pay the cost of synchronization (even though I haven't measured it), and in this particular case a data-race is safe.”
• Non-obvious sharing:
• “I thought this object wasn't shared between multiple threads.”
• Design changes.
• In the original design this object wasn't shared.
• But now it is, for some reason: design change, (bad) optimizations, singleton/caching.
fredag den 25. maj 12
![Page 18: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/18.jpg)
Is this in theory only?• Of course not!
• I've seen the “infinite loop” issue put down a cluster of production servers!
• Missing technical knowledge (e.g. JMM)
• Incorrect optimizations:
• “I really can't pay the cost of synchronization (even though I haven't measured it), and in this particular case a data-race is safe.”
• Non-obvious sharing:
• “I thought this object wasn't shared between multiple threads.”
• Design changes.
• In the original design this object wasn't shared.
• But now it is, for some reason: design change, (bad) optimizations, singleton/caching.
• Bad library/framework.
• The bug may not even be in your code!
fredag den 25. maj 12
![Page 19: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/19.jpg)
Some real-life bugs...
fredag den 25. maj 12
![Page 20: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/20.jpg)
Some real-life bugs...• Unsound optimization : MyFaces JSF Portlet Bridge
• Broken lazy initialization technique (+ another bug)
fredag den 25. maj 12
![Page 21: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/21.jpg)
Some real-life bugs...• Unsound optimization : MyFaces JSF Portlet Bridge
• Broken lazy initialization technique (+ another bug)
• IceFaces: bad reasoning & Design changes
• Store a mutable object (SwfLifecycleExecutor) in a map in application scope
• Each requests “initializes” it setting variables
• works in 1.8.0. broken 1.8.2
fredag den 25. maj 12
![Page 22: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/22.jpg)
Some real-life bugs...• Unsound optimization : MyFaces JSF Portlet Bridge
• Broken lazy initialization technique (+ another bug)
• IceFaces: bad reasoning & Design changes
• Store a mutable object (SwfLifecycleExecutor) in a map in application scope
• Each requests “initializes” it setting variables
• works in 1.8.0. broken 1.8.2
• Spring WebFlow: unintended sharing
• Storing non thread-safe object in application scope
• https://jira.springframework.org/browse/SWF-976
fredag den 25. maj 12
![Page 23: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/23.jpg)
Some real-life bugs...• Unsound optimization : MyFaces JSF Portlet Bridge
• Broken lazy initialization technique (+ another bug)
• IceFaces: bad reasoning & Design changes
• Store a mutable object (SwfLifecycleExecutor) in a map in application scope
• Each requests “initializes” it setting variables
• works in 1.8.0. broken 1.8.2
• Spring WebFlow: unintended sharing
• Storing non thread-safe object in application scope
• https://jira.springframework.org/browse/SWF-976
• ...
fredag den 25. maj 12
![Page 24: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/24.jpg)
Some real-life bugs...• Unsound optimization : MyFaces JSF Portlet Bridge
• Broken lazy initialization technique (+ another bug)
• IceFaces: bad reasoning & Design changes
• Store a mutable object (SwfLifecycleExecutor) in a map in application scope
• Each requests “initializes” it setting variables
• works in 1.8.0. broken 1.8.2
• Spring WebFlow: unintended sharing
• Storing non thread-safe object in application scope
• https://jira.springframework.org/browse/SWF-976
• ...
“This bug is very strange, because in single user development/testing everything works fine, but in production with more users, we got usually NullPointerException on bizzarre places in program like this...”
fredag den 25. maj 12
![Page 25: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/25.jpg)
Note
fredag den 25. maj 12
![Page 26: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/26.jpg)
Note• All these bugs have at their core
• Share mutable objects
• Low-level primitives: Threads, locks + JVM Memory Model
fredag den 25. maj 12
![Page 27: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/27.jpg)
Note• All these bugs have at their core
• Share mutable objects
• Low-level primitives: Threads, locks + JVM Memory Model
• Persistent data structures are Good!
• These bugs vanish when you restrict yourself to programming with immutable objects.
• But how to make that practical?
fredag den 25. maj 12
![Page 28: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/28.jpg)
Note• All these bugs have at their core
• Share mutable objects
• Low-level primitives: Threads, locks + JVM Memory Model
• Persistent data structures are Good!
• These bugs vanish when you restrict yourself to programming with immutable objects.
• But how to make that practical?
• It turns out, many immutable persistent data structures also are quite amenable to parallel processing.
fredag den 25. maj 12
![Page 29: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/29.jpg)
Agenda
• One-slide intro to Clojure
• Clojure Persistent Data Structures
• More details: PersistentVector, PersistentHashMap
• A look into the future: Parallelism with reducers
• Summary and references
fredag den 25. maj 12
![Page 30: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/30.jpg)
Clojure in one slide• Functional dynamic language
• Persistent data structures, pure functions, sequence library
• A unique programming & concurrency model
• State management constructs: var, ref, atom, agent
• On-the-fly & AOT compilation: JVM bytecode
• Deep two-way JVM interop.; embraces host
• A LISP family member
• Meta programming, closures, interactivity
fredag den 25. maj 12
![Page 31: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/31.jpg)
Remember, there is (much) more...
fredag den 25. maj 12
![Page 32: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/32.jpg)
Remember, there is (much) more...
• Records, types, protocols and polymorphism
• Multi methods
• Laziness
• Concurrency support
• Parallel programming support
• Macros and meta programming
• Numeric support and type hints
• JVM language interop
• ClojureScript
fredag den 25. maj 12
![Page 33: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/33.jpg)
Remember, there is (much) more...
• Records, types, protocols and polymorphism
• Multi methods
• Laziness
• Concurrency support
• Parallel programming support
• Macros and meta programming
• Numeric support and type hints
• JVM language interop
• ClojureScript
• Sequence library
• DSLs with Clojure
• Logic programming
• Meta data
• Persistent data structures
• Interactive/REPL-based programming
• Clojure CLR
• ...fredag den 25. maj 12
![Page 34: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/34.jpg)
Clojure Philosophy
fredag den 25. maj 12
![Page 35: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/35.jpg)
Clojure Philosophy• Background:
• Most programs could have dramatically less state than they do - we tend to introduce state just because it is the language default (and because we are trained to do so).
fredag den 25. maj 12
![Page 36: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/36.jpg)
Clojure Philosophy• Background:
• Most programs could have dramatically less state than they do - we tend to introduce state just because it is the language default (and because we are trained to do so).
• In Clojure
• We rarely use mutable objects, instead immutable data structures and pure functions.
• Explicitly mark the (few) parts of the program that have state - reference types.
• State-change to reference types
• is managed by Clojure (no manual locking).
• references atomically change from referring one immutable piece of data to another.
fredag den 25. maj 12
![Page 37: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/37.jpg)
Understanding Clojure’s Persistent Data Structures
fredag den 25. maj 12
![Page 38: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/38.jpg)
What are persistent data structures?
fredag den 25. maj 12
![Page 39: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/39.jpg)
What are persistent data structures?• Nothing to do with durability, i.e., saving to storage!
fredag den 25. maj 12
![Page 40: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/40.jpg)
What are persistent data structures?• Nothing to do with durability, i.e., saving to storage!
• Immutable
fredag den 25. maj 12
![Page 41: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/41.jpg)
What are persistent data structures?• Nothing to do with durability, i.e., saving to storage!
• Immutable
• There are efficient functions that
• take as input a PDS, and produce as output a ”variant” of the input.
• The input is still available after the operation and will retain its performance characteristics.
fredag den 25. maj 12
![Page 42: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/42.jpg)
What are persistent data structures?• Nothing to do with durability, i.e., saving to storage!
• Immutable
• There are efficient functions that
• take as input a PDS, and produce as output a ”variant” of the input.
• The input is still available after the operation and will retain its performance characteristics.
• For Clojure, performance characteristics usually within 1-4x of their mutable (Java) counterparts (for single ops).
• The input and output structures share most of their structure (which is efficient and safe).
fredag den 25. maj 12
![Page 43: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/43.jpg)
That almost feels like magic
fredag den 25. maj 12
![Page 44: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/44.jpg)
Data structures
Slide by Rich Hickey
fredag den 25. maj 12
![Page 45: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/45.jpg)
Understanding Persistent Vector
• How is a PersistentVector represented?
• How to do random access to the vector?
• How do we “add” elements to the vector?
fredag den 25. maj 12
![Page 46: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/46.jpg)
?public Object nth(int i){ if(i >= 0 && i < cnt) { if(i >= tailoff()) return tail[i & 0x01f];
Object[] arr = root; for(int level = shift; level > 0; level -= 5) arr = (Object[]) arr[(i >>> level) & 0x01f];
return arr[i & 0x01f]; } throw new IndexOutOfBoundsException();}
fredag den 25. maj 12
![Page 47: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/47.jpg)
Implemented as wide trees
fredag den 25. maj 12
![Page 48: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/48.jpg)
Bit partitioningVector index’es are ints (32 bit numbers)
Partition bit-representation in blocks of 5.
Each block corresponds to a level in the tree
(note, A block is also number in range [0,31])
fredag den 25. maj 12
![Page 49: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/49.jpg)
Bit partitioningVector index’es are ints (32 bit numbers)
Partition bit-representation in blocks of 5.
Each block corresponds to a level in the tree
(note, A block is also number in range [0,31])
Examples 1: [00][00000][00000][00000][00000][00000][00001]
234: [00][00000][00000][00000][00000][00111][01010](10,7)
1258: [00][00000][00000][00000][00001][00111][01010](10,7,1)
fredag den 25. maj 12
![Page 50: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/50.jpg)
Illustration
• 49: [...][00001][10001] (1 + 17)
fredag den 25. maj 12
![Page 51: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/51.jpg)
Illustration
• 49: [...][00001][10001] (1 + 17)
fredag den 25. maj 12
![Page 52: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/52.jpg)
Illustration
• 49: [...][00001][10001] (1 + 17)
fredag den 25. maj 12
![Page 53: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/53.jpg)
Illustration
• 49: [...][00001][10001] (1 + 17)
Level 5
Level 0
fredag den 25. maj 12
![Page 54: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/54.jpg)
Illustration
• 49: [...][00001][10001] (1 + 17)
Level 5
Level 0
(i >>> level) & 0x01f
fredag den 25. maj 12
![Page 55: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/55.jpg)
“Insertion” / Conjoin• Consider this situation again:
fredag den 25. maj 12
![Page 56: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/56.jpg)
“Insertion” / Conjoin• Consider this situation again:
fredag den 25. maj 12
![Page 57: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/57.jpg)
Easy-Peacy :)public Object nth(int i){ if(i >= 0 && i < cnt) { if(i >= tailoff()) return tail[i & 0x01f]; Object[] arr = root; for(int level = shift; level > 0; level -= 5) arr = (Object[]) arr[(i >>> level) & 0x01f]; return arr[i & 0x01f]; } throw new IndexOutOfBoundsException();}
1258: [00][00000][00000][00000][00001][00111][01010]
initial level (aka shift) is 10
fredag den 25. maj 12
![Page 58: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/58.jpg)
Adding a level
fredag den 25. maj 12
![Page 59: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/59.jpg)
What about HashMaps? PersistentHashMap• How is a PersistentHashMap represented?
• How to lookup the value for a key?
• How to “add” a new key-value pair?
• Note this presentation talks about version 1.2 of Clojure.
• There are changes in later versions.
• These are not essential to this talk.
fredag den 25. maj 12
![Page 60: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/60.jpg)
?? (clojure 1.2 version)static int mask(int hash, int shift){ return (hash >>> shift) & 0x01f;}
static int bitpos(int hash, int shift){ return 1 << mask(hash, shift);}
public LeafNode find(int hash, Object key){ int bit = bitpos(hash, shift); if((bitmap & bit) != 0) { return nodes[index(bit)].find(hash, key); } else return null;}
final int index(int bit){ return Integer.bitCount(bitmap & (bit - 1));}
fredag den 25. maj 12
![Page 61: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/61.jpg)
Slide by Rich Hickey
fredag den 25. maj 12
![Page 62: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/62.jpg)
Slide by Rich Hickey
fredag den 25. maj 12
![Page 63: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/63.jpg)
BitmapIndexedNode
fredag den 25. maj 12
![Page 64: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/64.jpg)
BitmapIndexedNode
• Holds an array of size < 32, pointing to children
fredag den 25. maj 12
![Page 65: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/65.jpg)
BitmapIndexedNode
• Holds an array of size < 32, pointing to children
• Hard part is to only use as much space as is needed:
• If node has n children, only use size n array;
• and, doing a lookup on a BitmapIndexedNode to find a child must be fast constant time
fredag den 25. maj 12
![Page 66: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/66.jpg)
BitmapIndexedNode
• Holds an array of size < 32, pointing to children
• Hard part is to only use as much space as is needed:
• If node has n children, only use size n array;
• and, doing a lookup on a BitmapIndexedNode to find a child must be fast constant time
• The trick is to find an efficiently computable function to map between a 5-bit number (i.e., a bit block) and index, 0 ≤ i < n in child array.
fredag den 25. maj 12
![Page 67: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/67.jpg)
BitmapIndexedNodeThe bitmap
fredag den 25. maj 12
![Page 68: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/68.jpg)
BitmapIndexedNodeThe bitmap
• Consider the mapping
• bitpos: [0, 31] => integer
• bitpos(n) = 2n (as bitpattern: 10n)
• e.g., 13 bitpos(13) = 00000010000000000000
fredag den 25. maj 12
![Page 69: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/69.jpg)
BitmapIndexedNodeThe bitmap
• Consider the mapping
• bitpos: [0, 31] => integer
• bitpos(n) = 2n (as bitpattern: 10n)
• e.g., 13 bitpos(13) = 00000010000000000000
• A bitmap is maintained which is a bit-pattern
• e.g., 00000100000001100010001000000001
fredag den 25. maj 12
![Page 70: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/70.jpg)
BitmapIndexedNodeThe bitmap
• Consider the mapping
• bitpos: [0, 31] => integer
• bitpos(n) = 2n (as bitpattern: 10n)
• e.g., 13 bitpos(13) = 00000010000000000000
• A bitmap is maintained which is a bit-pattern
• e.g., 00000100000001100010001000000001
• To check if an bitblock is in the array just match:
• 00000100000001100010001000000001
• 000000000000010000000000000
fredag den 25. maj 12
![Page 71: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/71.jpg)
Bitmap: indexing
fredag den 25. maj 12
![Page 72: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/72.jpg)
Bitmap: indexing• For a given bitmap, e.g.,
• 00000100000001100100001000000001
fredag den 25. maj 12
![Page 73: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/73.jpg)
Bitmap: indexing• For a given bitmap, e.g.,
• 00000100000001100100001000000001
• The index of an bit block, call bitpos first, e.g.:
• 00000000000001000000000000000000
fredag den 25. maj 12
![Page 74: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/74.jpg)
Bitmap: indexing• For a given bitmap, e.g.,
• 00000100000001100100001000000001
• The index of an bit block, call bitpos first, e.g.:
• 00000000000001000000000000000000
• Is the number of 1's below this bitpos, in the bitmap, in the above example: 4.
fredag den 25. maj 12
![Page 75: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/75.jpg)
Bitmap: indexing• For a given bitmap, e.g.,
• 00000100000001100100001000000001
• The index of an bit block, call bitpos first, e.g.:
• 00000000000001000000000000000000
• Is the number of 1's below this bitpos, in the bitmap, in the above example: 4.
• 00000000000001000000000000000000
fredag den 25. maj 12
![Page 76: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/76.jpg)
Bitmap: indexing• For a given bitmap, e.g.,
• 00000100000001100100001000000001
• The index of an bit block, call bitpos first, e.g.:
• 00000000000001000000000000000000
• Is the number of 1's below this bitpos, in the bitmap, in the above example: 4.
• 00000000000001000000000000000000
• 00000000000000111111111111111111 (index - 1)
fredag den 25. maj 12
![Page 77: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/77.jpg)
Bitmap: indexing• For a given bitmap, e.g.,
• 00000100000001100100001000000001
• The index of an bit block, call bitpos first, e.g.:
• 00000000000001000000000000000000
• Is the number of 1's below this bitpos, in the bitmap, in the above example: 4.
• 00000000000001000000000000000000
• 00000000000000111111111111111111 (index - 1)
• 00000000000000100100001000000001
fredag den 25. maj 12
![Page 78: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/78.jpg)
Bitmap: indexing• For a given bitmap, e.g.,
• 00000100000001100100001000000001
• The index of an bit block, call bitpos first, e.g.:
• 00000000000001000000000000000000
• Is the number of 1's below this bitpos, in the bitmap, in the above example: 4.
• 00000000000001000000000000000000
• 00000000000000111111111111111111 (index - 1)
• 00000000000000100100001000000001
• On many modern processors there is an instruction CTPOP/POPCNT (count population)
fredag den 25. maj 12
![Page 79: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/79.jpg)
Finding a nodestatic int mask(int hash, int shift){ return (hash >>> shift) & 0x01f;}
static int bitpos(int hash, int shift){ return 1 << mask(hash, shift);}
final int index(int bit){ return Integer.bitCount(bitmap & (bit - 1));}
public LeafNode find(int hash, Object key){ int bit = bitpos(hash, shift); if((bitmap & bit) != 0) { return nodes[index(bit)].find(hash, key); } else return null;}
fredag den 25. maj 12
![Page 80: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/80.jpg)
Structural Sharing
Past
Next
Slide by Rich Hickey
fredag den 25. maj 12
![Page 81: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/81.jpg)
Other benefits
fredag den 25. maj 12
![Page 82: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/82.jpg)
Other benefits• PersistentHashMap and PersistentVector amenable to
parallel processing.
• Divide and conquer (you already did half the work :)
• No special parallel collection types needed
fredag den 25. maj 12
![Page 83: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/83.jpg)
Other benefits• PersistentHashMap and PersistentVector amenable to
parallel processing.
• Divide and conquer (you already did half the work :)
• No special parallel collection types needed
• Clojure 1.5 will (likely) have a parallel processing function
• fold which takes a combining fn, a reducing fn and a collection
• Uses Java’s Fork/Join framework for parallel processing.
• Code has the same shape as existing (serial) clojure code
fredag den 25. maj 12
![Page 84: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/84.jpg)
Other benefits• PersistentHashMap and PersistentVector amenable to
parallel processing.
• Divide and conquer (you already did half the work :)
• No special parallel collection types needed
• Clojure 1.5 will (likely) have a parallel processing function
• fold which takes a combining fn, a reducing fn and a collection
• Uses Java’s Fork/Join framework for parallel processing.
• Code has the same shape as existing (serial) clojure code
• Live Example?
fredag den 25. maj 12
![Page 85: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/85.jpg)
Summary• For your own sake, please start doing functional programming.
Benefits
• Sanity
• Concurrency
• Parallelism
• (and it’s fun too)
• I recommend trying out Clojure :)
• Option clj-ds: https://github.com/krukow/clj-ds
• Port of Clojure’s persistent data structures to use (when stuck) with other JVM languages
• To be updated to support the upcoming parallel processing in Clojure
fredag den 25. maj 12
![Page 86: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/86.jpg)
References• Rich Hickey
• Are we there yet? http://www.infoq.com/presentations/Are-We-There-Yet-Rich-Hickey
• Clojure concurrency http://blip.tv/clojure/clojure-concurrency-819147
• http://clojure.com/blog/2012/05/08/reducers-a-library-and-model-for-collection-processing.html
• http://clojure.com/blog/2012/05/15/anatomy-of-reducer.html
• My Blog
• http://blog.higher-order.net/2009/02/01/understanding-clojures-persistentvector-implementation
• http://blog.higher-order.net/2009/09/08/understanding-clojures-persistenthashmap-deftwice/
fredag den 25. maj 12
![Page 87: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/87.jpg)
References• Chas Emerick, Brian Carper, &
Christophe Grand: Clojure Programming, 2012, O’Reilly
• clj-ds: https://github.com/krukow/clj-ds
• Port of Clojure’s persistent data structures to use (when stuck) with other JVM languages
• To be updated to support parallelism (via Clojure’s reducers)
• Conj Labs - 3 day Clojure training in Europe
• Lau Jensen & Karl Krukow
• Contact me [email protected] for details
fredag den 25. maj 12
![Page 88: Introduction to Clojure Concurrency (and data structures)gotocon.com/dl/goto-amsterdam-2012/slides/KarlKrukow_Introductio… · • Engineer at Trifork for about 6 years on Web, JavaScript,](https://reader034.vdocument.in/reader034/viewer/2022051512/603c9be780da53014d492678/html5/thumbnails/88.jpg)
Making app testing less painful...Please contact us with any questions:
[email protected]@lesspainful.com - iOS
[email protected] - Android
http://www.lesspainful.com
Questions?
fredag den 25. maj 12