gpars - the coolest bits

25
The coolest bits

Upload: artur-gajowy

Post on 12-Jul-2015

106 views

Category:

Software


3 download

TRANSCRIPT

Page 1: Gpars - the coolest bits

The coolest bits

Page 2: Gpars - the coolest bits

Why bother!?

Page 3: Gpars - the coolest bits

GParsGroovy Parallel Systems- multiple high-level parallelism abstractions- Java-usable- GAE-usable- part of Groovy SDK- open source (Apache 2 License)- Grooovy documentation

Page 4: Gpars - the coolest bits

GParsGroovy Parallel Systems

Team: Václav Pech, Dierk König, Russel Winder,Alex Tkachman, Paul King, Jon Kerridge,Rafał Sławik and others

Page 5: Gpars - the coolest bits

One Ring to rule them all...Parallelism approach in GPars (originally?) found in implemented using

Parallel collections Scala, Java jsr-166y / JDK5 executors

jsr-166y Fork/Join (sugar) Java jsr-166y

Actors Erlang, Scala java.util.concurrent

Software Transactional Memory Clojure, Scala Multiverse

Agents Clojure java.util.concurrent

CSP CSP :) Dataflow* or JCSP

Dataflow and extensions:Pipelines, Kanbanflow

Linda, Verilog, Agilent Vee, Scala, Unreal Engine

jsr-166y / JDK5 executors (?)

Composable asynchronous functions ? jsr-166y / JDK5 executors

Page 6: Gpars - the coolest bits
Page 7: Gpars - the coolest bits

Our focusParallelism approach in GPars (originally?) found in implemented using

Parallel collections Scala, Java jsr-166y / JDK5 executors

jsr-166y Fork/Join (sugar) Java jsr-166y

Actors Erlang, Scala java.util.concurrent

Software Transactional Memory Clojure, Scala Multiverse

Agents Clojure java.util.concurrent

CSP CSP :) Dataflow* or JCSP

Dataflow and extensions:Pipelines, Kanbanflow

Linda, Verilog, Agilent Vee, Scala, Unreal Engine

jsr-166y / JDK5 executors (?)

Composable asynchronous functions ? jsr-166y / JDK5 executors

Page 8: Gpars - the coolest bits

Demo time!Enabling GPars GDSL in Intellij IDEA

Page 9: Gpars - the coolest bits

Demo time!Parallel collections- withPool / enhancer- makeConcurrent / makeSequential /

asParallel- GParsPool + ParallelEnhancer /

GParsExecutorsPool + GParsExecutorsPoolEnhancer

- Map-Reduce (Fork / Join pool only)

Page 10: Gpars - the coolest bits

Things to consider / a warning...

In this approach, your iteration body must be:- stateless- side-effect free- constrained to the current elementElse - welcome to concurrency hell...

Page 11: Gpars - the coolest bits

Demo time!Agents- updating immutable / mutable- syntax equivalents for .send- copy method- parallel behaviour- error handling- parallel groups, daemon threads & shutting

down

Page 12: Gpars - the coolest bits

Agents - what else is there- listeners- validators & validation gotchas- non-closure sends- prefer composition over inheritance- sendAndAwait

Page 13: Gpars - the coolest bits

Agents - Q & AQuestions?- what if I want to check a condition

before updating state?- how do I 'lock' the state?

Page 14: Gpars - the coolest bits

Composable Asynchronous Functions- @AsyncFun / asyncFun()- async funs return DataflowVariable-s- blocking @AsyncFun-s for convenience- rest of the docs: instantiation variants,

pool configuration

Demo time!

Page 15: Gpars - the coolest bits

Bonus: on parallelism flavours

Page 16: Gpars - the coolest bits

Choosing an approachTask parallelism

Independent Async closures, Futures, whatever...

Dependent Random / Unknown Dataflow

Functional / Linear Composable asynchronous functions, Dataflow tasks, CSP

Hierarchical / Recursive Fork / Join

Shared state Agents, STM

Page 17: Gpars - the coolest bits

Choosing an approachData parallelism

Geometrical / Linear Parallel collections

Hierarchical / Recursive Fork / Join

Streamed (regular) Pipeline DSL, Dataflow, CSP

Event-driven (irregular) Actors

Page 18: Gpars - the coolest bits

From GPARS User Guide

Page 19: Gpars - the coolest bits

TL;DRParallel collectionsAsync functionsDataflowAgentsMessage passing - CSP - Dataflow - Actors

Page 20: Gpars - the coolest bits

Also: Bashing actors a little :)

Actors are overratedDierk König

Page 21: Gpars - the coolest bits

Also: Bashing actors a little :)Why there were no Actors in Clojure*● complex● no efficient sharing even for immutable structures● reduce flexibility● do not really distribute well from single to many

machines

* a purposefully exaggerating summary by me

Page 23: Gpars - the coolest bits

Recap- GPars is quite a lib :)- Be sure to try Dataflows- Actors are overrated ;)

Page 24: Gpars - the coolest bits

Questions?

Page 25: Gpars - the coolest bits

Thank you!