scala%parallel%collec+ons%days2011.scala-lang.org/sites/days2011/files/29. parallel...
TRANSCRIPT
![Page 1: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/1.jpg)
Scala Parallel Collec+ons Aleksandar Prokopec
EPFL
![Page 2: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/2.jpg)
![Page 3: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/3.jpg)
Scala collec+ons
for { s <- surnames n <- names if s endsWith n } yield (n, s)
McDonald
![Page 4: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/4.jpg)
Scala collec+ons
for { s <- surnames n <- names if s endsWith n } yield (n, s)
1040 ms
![Page 5: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/5.jpg)
![Page 6: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/6.jpg)
Scala parallel collec+ons
for { s <- surnames n <- names if s endsWith n } yield (n, s)
![Page 7: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/7.jpg)
Scala parallel collec+ons
for { s <- surnames.par n <- names.par if s endsWith n } yield (n, s)
![Page 8: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/8.jpg)
Scala parallel collec+ons
for { s <- surnames.par n <- names.par if s endsWith n } yield (n, s)
2 cores
575 ms
![Page 9: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/9.jpg)
Scala parallel collec+ons
for { s <- surnames.par n <- names.par if s endsWith n } yield (n, s)
4 cores
305 ms
![Page 10: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/10.jpg)
for comprehensions
surnames.par.flatMap { s => names.par .filter(n => s endsWith n) .map(n => (n, s)) }
![Page 11: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/11.jpg)
for comprehensions nested parallelized bulk opera+ons
surnames.par.flatMap { s => names.par .filter(n => s endsWith n) .map(n => (n, s)) }
![Page 12: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/12.jpg)
Nested parallelism
![Page 13: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/13.jpg)
Nested parallelism parallel within parallel
composi+on
surnames.par.flatMap { s => surnameToCollection(s) // may invoke parallel ops }
![Page 14: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/14.jpg)
Nested parallelism going recursive
def vowel(c: Char): Boolean = ...
![Page 15: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/15.jpg)
Nested parallelism going recursive
def vowel(c: Char): Boolean = ... def gen(n: Int, acc: Seq[String]): Seq[String] = if (n == 0) acc
![Page 16: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/16.jpg)
Nested parallelism going recursive
def vowel(c: Char): Boolean = ... def gen(n: Int, acc: Seq[String]): Seq[String] = if (n == 0) acc else for (s <- gen(n - 1, acc); c <- 'a' to 'z') yield
recursive algorithms
![Page 17: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/17.jpg)
Nested parallelism going recursive
def vowel(c: Char): Boolean = ... def gen(n: Int, acc: Seq[String]): Seq[String] = if (n == 0) acc else for (s <- gen(n - 1, acc); c <- 'a' to 'z') yield if (s.length == 0) s + c
![Page 18: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/18.jpg)
Nested parallelism going recursive
def vowel(c: Char): Boolean = ... def gen(n: Int, acc: Seq[String]): Seq[String] = if (n == 0) acc else for (s <- gen(n - 1, acc); c <- 'a' to 'z') yield if (s.length == 0) s + c else if (vowel(s.last) && !vowel(c)) s + c else if (!vowel(s.last) && vowel(c)) s + c
![Page 19: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/19.jpg)
Nested parallelism going recursive
def vowel(c: Char): Boolean = ... def gen(n: Int, acc: Seq[String]): Seq[String] = if (n == 0) acc else for (s <- gen(n - 1, acc); c <- 'a' to 'z') yield if (s.length == 0) s + c else if (vowel(s.last) && !vowel(c)) s + c else if (!vowel(s.last) && vowel(c)) s + c else s gen(5, Array(""))
![Page 20: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/20.jpg)
Nested parallelism going recursive
def vowel(c: Char): Boolean = ... def gen(n: Int, acc: Seq[String]): Seq[String] = if (n == 0) acc else for (s <- gen(n - 1, acc); c <- 'a' to 'z') yield if (s.length == 0) s + c else if (vowel(s.last) && !vowel(c)) s + c else if (!vowel(s.last) && vowel(c)) s + c else s gen(5, Array(""))
1545 ms
![Page 21: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/21.jpg)
Nested parallelism going recursive
def vowel(c: Char): Boolean = ... def gen(n: Int, acc: ParSeq[String]): ParSeq[String] = if (n == 0) acc else for (s <- gen(n - 1, acc); c <- 'a' to 'z') yield if (s.length == 0) s + c else if (vowel(s.last) && !vowel(c)) s + c else if (!vowel(s.last) && vowel(c)) s + c else s gen(5, ParArray(""))
![Page 22: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/22.jpg)
Nested parallelism going recursive
def vowel(c: Char): Boolean = ... def gen(n: Int, acc: ParSeq[String]): ParSeq[String] = if (n == 0) acc else for (s <- gen(n - 1, acc); c <- 'a' to 'z') yield if (s.length == 0) s + c else if (vowel(s.last) && !vowel(c)) s + c else if (!vowel(s.last) && vowel(c)) s + c else s gen(5, ParArray("")) 1 core
1575 ms
![Page 23: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/23.jpg)
Nested parallelism going recursive
def vowel(c: Char): Boolean = ... def gen(n: Int, acc: ParSeq[String]): ParSeq[String] = if (n == 0) acc else for (s <- gen(n - 1, acc); c <- 'a' to 'z') yield if (s.length == 0) s + c else if (vowel(s.last) && !vowel(c)) s + c else if (!vowel(s.last) && vowel(c)) s + c else s gen(5, ParArray("")) 2 cores
809 ms
![Page 24: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/24.jpg)
Nested parallelism going recursive
def vowel(c: Char): Boolean = ... def gen(n: Int, acc: ParSeq[String]): ParSeq[String] = if (n == 0) acc else for (s <- gen(n - 1, acc); c <- 'a' to 'z') yield if (s.length == 0) s + c else if (vowel(s.last) && !vowel(c)) s + c else if (!vowel(s.last) && vowel(c)) s + c else s gen(5, ParArray("")) 4 cores
530 ms
![Page 25: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/25.jpg)
So, I just use par and I’m home free?
![Page 26: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/26.jpg)
How to think parallel
![Page 27: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/27.jpg)
Character count use case for foldLeQ
val txt: String = ... txt.foldLeft(0) { case (a, ‘ ‘) => a case (a, c) => a + 1 }
![Page 28: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/28.jpg)
6 5 4 3 2 1 0
Character count use case for foldLeQ
txt.foldLeft(0) { case (a, ‘ ‘) => a case (a, c) => a + 1 }
going leQ to right -‐ not parallelizable!
A B C D E F
_ + 1
![Page 29: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/29.jpg)
Character count use case for foldLeQ
txt.foldLeft(0) { case (a, ‘ ‘) => a case (a, c) => a + 1 }
going leQ to right – not really necessary
3 2 1 0 A B C
_ + 1
3 2 1 0 D E F
_ + 1
_ + _ 6
![Page 30: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/30.jpg)
Character count in parallel
txt.fold(0) { case (a, ‘ ‘) => a case (a, c) => a + 1 }
![Page 31: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/31.jpg)
Character count in parallel
txt.fold(0) { case (a, ‘ ‘) => a case (a, c) => a + 1 }
3 2 1 1 A B C
_ + 1
3 2 1 1 A B C
: (Int, Char) => Int
![Page 32: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/32.jpg)
Character count fold not applicable
txt.fold(0) { case (a, ‘ ‘) => a case (a, c) => a + 1 }
3 2 1 3 A B C
_ + _ 3 3
3 2 1 3 A B C
! (Int, Int) => Int
![Page 33: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/33.jpg)
Character count use case for aggregate
txt.aggregate(0)({ case (a, ‘ ‘) => a case (a, c) => a + 1 }, _ + _)
![Page 34: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/34.jpg)
3 2 1 1 A B C
Character count use case for aggregate
txt.aggregate(0)({ case (a, ‘ ‘) => a case (a, c) => a + 1 }, _ + _)
_ + _ 3 3
3 2 1 3 A B C
_ + 1
![Page 35: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/35.jpg)
Character count use case for aggregate
aggrega+on element
3 2 1 1 A B C
_ + _ 3 3
3 2 1 3 A B C
txt.aggregate(0)({ case (a, ‘ ‘) => a case (a, c) => a + 1 }, _ + _)
B
_ + 1
![Page 36: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/36.jpg)
Character count use case for aggregate
aggrega+on aggrega+on aggrega+on element
3 2 1 1 A B C
_ + _ 3 3
3 2 1 3 A B C
txt.aggregate(0)({ case (a, ‘ ‘) => a case (a, c) => a + 1 }, _ + _)
B
_ + 1
![Page 37: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/37.jpg)
Word count another use case for foldLeQ
txt.foldLeft((0, true)) { case ((wc, _), ' ') => (wc, true) case ((wc, true), x) => (wc + 1, false) case ((wc, false), x) => (wc, false) }
![Page 38: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/38.jpg)
Word count ini+al accumula+on
txt.foldLeft((0, true)) { case ((wc, _), ' ') => (wc, true) case ((wc, true), x) => (wc + 1, false) case ((wc, false), x) => (wc, false) }
0 words so far last character was a space
“Folding me softly.”
![Page 39: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/39.jpg)
Word count a space
txt.foldLeft((0, true)) { case ((wc, _), ' ') => (wc, true) case ((wc, true), x) => (wc + 1, false) case ((wc, false), x) => (wc, false) }
“Folding me softly.”
last seen character is a space
![Page 40: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/40.jpg)
Word count a non space
txt.foldLeft((0, true)) { case ((wc, _), ' ') => (wc, true) case ((wc, true), x) => (wc + 1, false) case ((wc, false), x) => (wc, false) }
“Folding me softly.”
last seen character was a space – a new word
![Page 41: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/41.jpg)
Word count a non space
txt.foldLeft((0, true)) { case ((wc, _), ' ') => (wc, true) case ((wc, true), x) => (wc + 1, false) case ((wc, false), x) => (wc, false) }
“Folding me softly.”
last seen character wasn’t a space – no new word
![Page 42: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/42.jpg)
Word count in parallel
“softly.“ “Folding me “
P1 P2
![Page 43: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/43.jpg)
Word count in parallel
“softly.“ “Folding me “
wc = 2; rs = 1 wc = 1; ls = 0
P1 P2
![Page 44: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/44.jpg)
Word count in parallel
“softly.“ “Folding me “
wc = 2; rs = 1 wc = 1; ls = 0 wc = 3
P1 P2
![Page 45: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/45.jpg)
Word count must assume arbitrary par++ons
“g me softly.“ “Foldin“
wc = 1; rs = 0 wc = 3; ls = 0
P1 P2
![Page 46: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/46.jpg)
Word count must assume arbitrary par++ons
“g me softly.“ “Foldin“
wc = 1; rs = 0 wc = 3; ls = 0
P1 P2
wc = 3
![Page 47: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/47.jpg)
Word count ini+al aggrega+on
txt.par.aggregate((0, 0, 0))
![Page 48: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/48.jpg)
Word count ini+al aggrega+on
txt.par.aggregate((0, 0, 0))
# spaces on the leQ # spaces on the right #words
![Page 49: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/49.jpg)
Word count ini+al aggrega+on
txt.par.aggregate((0, 0, 0))
# spaces on the leQ # spaces on the right #words
””
![Page 50: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/50.jpg)
Word count aggrega+on aggrega+on
... }, { case ((0, 0, 0), res) => res case (res, (0, 0, 0)) => res
““ “Folding me“ “softly.“ ““
![Page 51: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/51.jpg)
Word count aggrega+on aggrega+on
... }, { case ((0, 0, 0), res) => res case (res, (0, 0, 0)) => res case ((lls, lwc, 0), (0, rwc, rrs)) => (lls, lwc + rwc - 1, rrs)
“e softly.“ “Folding m“
![Page 52: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/52.jpg)
Word count aggrega+on aggrega+on
... }, { case ((0, 0, 0), res) => res case (res, (0, 0, 0)) => res case ((lls, lwc, 0), (0, rwc, rrs)) => (lls, lwc + rwc - 1, rrs) case ((lls, lwc, _), (_, rwc, rrs)) => (lls, lwc + rwc, rrs)
“ softly.“ “Folding me”
![Page 53: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/53.jpg)
Word count aggrega+on element
txt.par.aggregate((0, 0, 0))({ case ((ls, 0, _), ' ') => (ls + 1, 0, ls + 1)
”_”
0 words and a space – add one more space each side
![Page 54: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/54.jpg)
Word count aggrega+on element
txt.par.aggregate((0, 0, 0))({ case ((ls, 0, _), ' ') => (ls + 1, 0, ls + 1) case ((ls, 0, _), c) => (ls, 1, 0)
” m”
0 words and a non-‐space – one word, no spaces on the right side
![Page 55: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/55.jpg)
Word count aggrega+on element
txt.par.aggregate((0, 0, 0))({ case ((ls, 0, _), ' ') => (ls + 1, 0, ls + 1) case ((ls, 0, _), c) => (ls, 1, 0) case ((ls, wc, rs), ' ') => (ls, wc, rs + 1)
” me_”
nonzero words and a space – one more space on the right side
![Page 56: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/56.jpg)
Word count aggrega+on element
txt.par.aggregate((0, 0, 0))({ case ((ls, 0, _), ' ') => (ls + 1, 0, ls + 1) case ((ls, 0, _), c) => (ls, 1, 0) case ((ls, wc, rs), ' ') => (ls, wc, rs + 1) case ((ls, wc, 0), c) => (ls, wc, 0)
” me sof”
nonzero words, last non-‐space and current non-‐space – no change
![Page 57: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/57.jpg)
Word count aggrega+on element
txt.par.aggregate((0, 0, 0))({ case ((ls, 0, _), ' ') => (ls + 1, 0, ls + 1) case ((ls, 0, _), c) => (ls, 1, 0) case ((ls, wc, rs), ' ') => (ls, wc, rs + 1) case ((ls, wc, 0), c) => (ls, wc, 0) case ((ls, wc, rs), c) => (ls, wc + 1, 0)
” me s”
nonzero words, last space and current non-‐space – one more word
![Page 58: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/58.jpg)
Word count in parallel
txt.par.aggregate((0, 0, 0))({ case ((ls, 0, _), ' ') => (ls + 1, 0, ls + 1) case ((ls, 0, _), c) => (ls, 1, 0) case ((ls, wc, rs), ' ') => (ls, wc, rs + 1) case ((ls, wc, 0), c) => (ls, wc, 0) case ((ls, wc, rs), c) => (ls, wc + 1, 0) }, { case ((0, 0, 0), res) => res case (res, (0, 0, 0)) => res case ((lls, lwc, 0), (0, rwc, rrs)) => (lls, lwc + rwc - 1, rrs) case ((lls, lwc, _), (_, rwc, rrs)) => (lls, lwc + rwc, rrs) })
![Page 59: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/59.jpg)
Word count using parallel strings?
txt.par.aggregate((0, 0, 0))({ case ((ls, 0, _), ' ') => (ls + 1, 0, ls + 1) case ((ls, 0, _), c) => (ls, 1, 0) case ((ls, wc, rs), ' ') => (ls, wc, rs + 1) case ((ls, wc, 0), c) => (ls, wc, 0) case ((ls, wc, rs), c) => (ls, wc + 1, 0) }, { case ((0, 0, 0), res) => res case (res, (0, 0, 0)) => res case ((lls, lwc, 0), (0, rwc, rrs)) => (lls, lwc + rwc - 1, rrs) case ((lls, lwc, _), (_, rwc, rrs)) => (lls, lwc + rwc, rrs) })
![Page 60: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/60.jpg)
Word count string not really parallelizable
scala> (txt: String).par
![Page 61: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/61.jpg)
Word count string not really parallelizable
scala> (txt: String).par collection.parallel.ParSeq[Char] = ParArray(…)
![Page 62: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/62.jpg)
Word count string not really parallelizable
scala> (txt: String).par collection.parallel.ParSeq[Char] = ParArray(…)
different internal representa+on!
![Page 63: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/63.jpg)
Word count string not really parallelizable
scala> (txt: String).par collection.parallel.ParSeq[Char] = ParArray(…)
different internal representa+on!
ParArray
![Page 64: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/64.jpg)
Word count string not really parallelizable
scala> (txt: String).par collection.parallel.ParSeq[Char] = ParArray(…)
different internal representa+on!
ParArray
copy string contents into an array
![Page 65: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/65.jpg)
Conversions going parallel
// par is efficient – no copying mutable.{Array, ArrayBuffer, ArraySeq}
mutable.{HashMap, HashSet} immutable.{Vector, Range} immutable.{HashMap, HashSet}
![Page 66: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/66.jpg)
Conversions going parallel
// par is efficient – no copying mutable.{Array, ArrayBuffer, ArraySeq}
mutable.{HashMap, HashSet} immutable.{Vector, Range} immutable.{HashMap, HashSet}
most other collec+ons construct a new parallel collec+on!
![Page 67: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/67.jpg)
Conversions going parallel
sequen&al parallel
Array, ArrayBuffer, ArraySeq mutable.ParArray
mutable.HashMap mutable.ParHashMap
mutable.HashSet mutable.ParHashSet
immutable.Vector immutable.ParVector
immutable.Range immutable.ParRange
immutable.HashMap immutable.ParHashMap
immutable.HashSet immutable.ParHashSet
![Page 68: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/68.jpg)
Custom collec+ons
![Page 69: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/69.jpg)
Custom collec+on
class ParString(val str: String)
![Page 70: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/70.jpg)
Custom collec+on
class ParString(val str: String) extends parallel.immutable.ParSeq[Char] {
![Page 71: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/71.jpg)
Custom collec+on
class ParString(val str: String) extends parallel.immutable.ParSeq[Char] { def apply(i: Int) = str.charAt(i) def length = str.length
![Page 72: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/72.jpg)
Custom collec+on
class ParString(val str: String) extends parallel.immutable.ParSeq[Char] { def apply(i: Int) = str.charAt(i) def length = str.length def seq = new WrappedString(str)
![Page 73: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/73.jpg)
Custom collec+on
class ParString(val str: String) extends parallel.immutable.ParSeq[Char] { def apply(i: Int) = str.charAt(i) def length = str.length def seq = new WrappedString(str) def splitter: Splitter[Char]
![Page 74: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/74.jpg)
Custom collec+on
class ParString(val str: String) extends parallel.immutable.ParSeq[Char] { def apply(i: Int) = str.charAt(i) def length = str.length def seq = new WrappedString(str) def splitter = new ParStringSplitter(0, str.length)
![Page 75: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/75.jpg)
Custom collec+on spli_er defini+on
class ParStringSplitter(var i: Int, len: Int) extends Splitter[Char] {
![Page 76: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/76.jpg)
Custom collec+on spli_ers are iterators
class ParStringSplitter(i: Int, len: Int) extends Splitter[Char] { def hasNext = i < len def next = { val r = str.charAt(i) i += 1 r }
![Page 77: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/77.jpg)
Custom collec+on spli_ers must be duplicated
... def dup = new ParStringSplitter(i, len)
![Page 78: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/78.jpg)
Custom collec+on spli_ers know how many elements remain
... def dup = new ParStringSplitter(i, len) def remaining = len - i
![Page 79: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/79.jpg)
Custom collec+on spli_ers can be split
... def psplit(sizes: Int*): Seq[ParStringSplitter] = { val splitted = new ArrayBuffer[ParStringSplitter] for (sz <- sizes) { val next = (i + sz) min ntl splitted += new ParStringSplitter(i, next) i = next } splitted }
![Page 80: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/80.jpg)
Word count now with parallel strings
new ParString(txt).aggregate((0, 0, 0))({ case ((ls, 0, _), ' ') => (ls + 1, 0, ls + 1) case ((ls, 0, _), c) => (ls, 1, 0) case ((ls, wc, rs), ' ') => (ls, wc, rs + 1) case ((ls, wc, 0), c) => (ls, wc, 0) case ((ls, wc, rs), c) => (ls, wc + 1, 0) }, { case ((0, 0, 0), res) => res case (res, (0, 0, 0)) => res case ((lls, lwc, 0), (0, rwc, rrs)) => (lls, lwc + rwc - 1, rrs) case ((lls, lwc, _), (_, rwc, rrs)) => (lls, lwc + rwc, rrs) })
![Page 81: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/81.jpg)
Word count performance
txt.foldLeft((0, true)) { case ((wc, _), ' ') => (wc, true) case ((wc, true), x) => (wc + 1, false) case ((wc, false), x) => (wc, false) }
new ParString(txt).aggregate((0, 0, 0))({ case ((ls, 0, _), ' ') => (ls + 1, 0, ls + 1) case ((ls, 0, _), c) => (ls, 1, 0) case ((ls, wc, rs), ' ') => (ls, wc, rs + 1) case ((ls, wc, 0), c) => (ls, wc, 0) case ((ls, wc, rs), c) => (ls, wc + 1, 0) }, { case ((0, 0, 0), res) => res case (res, (0, 0, 0)) => res case ((lls, lwc, 0), (0, rwc, rrs)) => (lls, lwc + rwc - 1, rrs) case ((lls, lwc, _), (_, rwc, rrs)) => (lls, lwc + rwc, rrs) })
100 ms
cores: 1 2 4 +me: 137 ms 70 ms 35 ms
![Page 82: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/82.jpg)
Hierarchy
GenTraversable
GenIterable
GenSeq
Traversable
Iterable
Seq
ParIterable
ParSeq
![Page 83: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/83.jpg)
Hierarchy
def nonEmpty(sq: Seq[String]) = { val res = new mutable.ArrayBuffer[String]() for (s <- sq) { if (s.nonEmpty) res += s } res }
![Page 84: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/84.jpg)
Hierarchy
def nonEmpty(sq: ParSeq[String]) = { val res = new mutable.ArrayBuffer[String]() for (s <- sq) { if (s.nonEmpty) res += s } res }
![Page 85: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/85.jpg)
Hierarchy
def nonEmpty(sq: ParSeq[String]) = { val res = new mutable.ArrayBuffer[String]() for (s <- sq) { if (s.nonEmpty) res += s } res }
side-‐effects! ArrayBuffer is not synchronized!
![Page 86: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/86.jpg)
Hierarchy
def nonEmpty(sq: ParSeq[String]) = { val res = new mutable.ArrayBuffer[String]() for (s <- sq) { if (s.nonEmpty) res += s } res }
side-‐effects! ArrayBuffer is not synchronized!
ParSeq
Seq
![Page 87: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/87.jpg)
Hierarchy
def nonEmpty(sq: GenSeq[String]) = { val res = new mutable.ArrayBuffer[String]() for (s <- sq) { if (s.nonEmpty) res.synchronized { res += s } } res }
![Page 88: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/88.jpg)
Thank you!
Examples at: git://github.com/axel22/sd.git
![Page 89: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/89.jpg)
Accessors vs. transformers some methods need more than just spli_ers
foreach, reduce, find, sameElements, indexOf, corresponds, forall, exists, max, min, sum, count, …
map, flatMap, filter, partition, ++, take, drop, span, zip, patch, padTo, …
![Page 90: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/90.jpg)
Accessors vs. transformers some methods need more than just spli_ers
foreach, reduce, find, sameElements, indexOf, corresponds, forall, exists, max, min, sum, count, …
map, flatMap, filter, partition, ++, take, drop, span, zip, patch, padTo, …
These return collec+ons!
![Page 91: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/91.jpg)
Accessors vs. transformers some methods need more than just spli_ers
foreach, reduce, find, sameElements, indexOf, corresponds, forall, exists, max, min, sum, count, …
map, flatMap, filter, partition, ++, take, drop, span, zip, patch, padTo, …
Sequen+al collec+ons – builders
![Page 92: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/92.jpg)
Accessors vs. transformers some methods need more than just spli_ers
foreach, reduce, find, sameElements, indexOf, corresponds, forall, exists, max, min, sum, count, …
map, flatMap, filter, partition, ++, take, drop, span, zip, patch, padTo, …
Sequen+al collec+ons – builders Parallel collec+ons – combiners
![Page 93: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/93.jpg)
Builders building a sequen+al collec+on
1 2 3 4 5 6 7 Nil 2 4 6
Nil
ListBuilder
+= += +=
result
![Page 94: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/94.jpg)
Combiners building parallel collec+ons
trait Combiner[-Elem, +To] extends Builder[Elem, To] { def combine[N <: Elem, NewTo >: To] (other: Combiner[N, NewTo]): Combiner[N, NewTo] }
![Page 95: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/95.jpg)
Combiners building parallel collec+ons
trait Combiner[-Elem, +To] extends Builder[Elem, To] { def combine[N <: Elem, NewTo >: To] (other: Combiner[N, NewTo]): Combiner[N, NewTo] }
Combiner Combiner Combiner
![Page 96: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/96.jpg)
Combiners building parallel collec+ons
trait Combiner[-Elem, +To] extends Builder[Elem, To] { def combine[N <: Elem, NewTo >: To] (other: Combiner[N, NewTo]): Combiner[N, NewTo] }
either use an efficient merge opera+on or do lazy evalua+on
![Page 97: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/97.jpg)
Parallel arrays
1, 2, 3, 4 5, 6, 7, 8 2, 4 6, 8 3, 1, 8, 0 2, 2, 1, 9 8, 0 2, 2
merge merge
merge copy
allocate
2 4 6 8 8 0 2 2
![Page 98: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/98.jpg)
Parallel hash tables
ParHashMap
![Page 99: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/99.jpg)
Parallel hash tables
ParHashMap 0 1 2 4 5 7 8 9
e.g. calling filter
![Page 100: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/100.jpg)
Parallel hash tables
ParHashMap 0 1 2 4 5 7 8 9
ParHashCombiner ParHashCombiner
e.g. calling filter
0 5 1 7 9 4
![Page 101: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/101.jpg)
Parallel hash tables
ParHashMap 0 1 2 4 5 7 8 9
ParHashCombiner
0 1 4
ParHashCombiner
5 7 9
![Page 102: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/102.jpg)
Parallel hash tables
ParHashMap 0 1 2 4 5 7 8 9
ParHashCombiner
0 1 4
ParHashCombiner
5 9
5 7 0 1 4
7
9
![Page 103: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/103.jpg)
Parallel hash tables
ParHashMap
ParHashCombiner ParHashCombiner
How to merge?
5 7 0 1 4 9
![Page 104: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/104.jpg)
5 7 8 9 1 4 0
Parallel hash tables
buckets! ParHashCombiner ParHashCombiner
0 1 4 9 7 5
ParHashMap 2 0 = 00002
1 = 00012 4 = 01002
![Page 105: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/105.jpg)
Parallel hash tables
ParHashCombiner ParHashCombiner
0
1
4 9
7
5
combine
![Page 106: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/106.jpg)
Parallel hash tables
ParHashCombiner ParHashCombiner
9
7
5 0
1
4
ParHashCombiner
no copying!
![Page 107: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/107.jpg)
Parallel hash tables
9
7
5
0
1
4
ParHashCombiner
![Page 108: Scala%Parallel%Collec+ons%days2011.scala-lang.org/sites/days2011/files/29. Parallel Collections.pdfScala parallelcollecons for { s](https://reader036.vdocument.in/reader036/viewer/2022081522/5f75504b0068f516975e59b9/html5/thumbnails/108.jpg)
Parallel hash tables
9 7 5 0 1 4
ParHashMap