java 8 parallel stream internals (part 6)schmidt/cs891f/2018-pdfs/12-parallelstrea… · •return...
TRANSCRIPT
![Page 1: Java 8 Parallel Stream Internals (Part 6)schmidt/cs891f/2018-PDFs/12-parallelstrea… · •return ArrayList::new Implementing Concurrent & Non-Concurrent Collectors A non-concurrent](https://reader030.vdocument.in/reader030/viewer/2022040309/5f0bdda47e708231d432991a/html5/thumbnails/1.jpg)
Java 8 Parallel Stream Internals
(Part 6)
Douglas C. [email protected]
www.dre.vanderbilt.edu/~schmidt
Professor of Computer Science
Institute for Software
Integrated Systems
Vanderbilt University
Nashville, Tennessee, USA
![Page 2: Java 8 Parallel Stream Internals (Part 6)schmidt/cs891f/2018-PDFs/12-parallelstrea… · •return ArrayList::new Implementing Concurrent & Non-Concurrent Collectors A non-concurrent](https://reader030.vdocument.in/reader030/viewer/2022040309/5f0bdda47e708231d432991a/html5/thumbnails/2.jpg)
2
• Understand parallel stream internals, e.g.
• Know what can change & what can’t
• Partition a data source into “chunks”
• Process chunks in parallel
• Configure the Java 8 parallel stream common fork-join pool
• Avoid pool starvation & improve performance w/ManagedBlocker
• Perform a reduction that combines partial results into a single result
• Learn to implement concurrent & non-concurrent collectors
Learning Objectives in this Part of the Lesson
Processsequentially
Processsequentially
Processsequentially
Processsequentially
InputString1.1 InputString1.2 InputString2.1 InputString2.2
InputString1 InputString2
trySplit()
InputString
trySplit() trySplit()
accumulate() accumulate()
accumulate()
ConcurrentResult Container
![Page 3: Java 8 Parallel Stream Internals (Part 6)schmidt/cs891f/2018-PDFs/12-parallelstrea… · •return ArrayList::new Implementing Concurrent & Non-Concurrent Collectors A non-concurrent](https://reader030.vdocument.in/reader030/viewer/2022040309/5f0bdda47e708231d432991a/html5/thumbnails/3.jpg)
3
Implementing Concurrent & Non-Concurrent Collectors
![Page 4: Java 8 Parallel Stream Internals (Part 6)schmidt/cs891f/2018-PDFs/12-parallelstrea… · •return ArrayList::new Implementing Concurrent & Non-Concurrent Collectors A non-concurrent](https://reader030.vdocument.in/reader030/viewer/2022040309/5f0bdda47e708231d432991a/html5/thumbnails/4.jpg)
4
• Collector defines an interface whose implementations can accumulate input elements in a mutable result container
See docs.oracle.com/javase/8/docs/api/java/util/stream/Collector.html
Implementing Concurrent & Non-Concurrent Collectors
![Page 5: Java 8 Parallel Stream Internals (Part 6)schmidt/cs891f/2018-PDFs/12-parallelstrea… · •return ArrayList::new Implementing Concurrent & Non-Concurrent Collectors A non-concurrent](https://reader030.vdocument.in/reader030/viewer/2022040309/5f0bdda47e708231d432991a/html5/thumbnails/5.jpg)
5
• Collector implementations can either benon-concurrent or concurrent based on their characteristics
See docs.oracle.com/javase/8/docs/api/java/util/stream/Collector.Characteristics.html
Implementing Concurrent & Non-Concurrent Collectors
![Page 6: Java 8 Parallel Stream Internals (Part 6)schmidt/cs891f/2018-PDFs/12-parallelstrea… · •return ArrayList::new Implementing Concurrent & Non-Concurrent Collectors A non-concurrent](https://reader030.vdocument.in/reader030/viewer/2022040309/5f0bdda47e708231d432991a/html5/thumbnails/6.jpg)
6
• Collector implementations can either benon-concurrent or concurrent based on their characteristics
• This distinction is only relevant forparallel streams
See “Overview of Java 8 Streams (Part 4)” for non-concurrent collector implementation
Implementing Concurrent & Non-Concurrent Collectors
filter(not(this::urlCached))
collect(toList())
map(this::downloadImage)
flatMap(this::applyFilters)
…
![Page 7: Java 8 Parallel Stream Internals (Part 6)schmidt/cs891f/2018-PDFs/12-parallelstrea… · •return ArrayList::new Implementing Concurrent & Non-Concurrent Collectors A non-concurrent](https://reader030.vdocument.in/reader030/viewer/2022040309/5f0bdda47e708231d432991a/html5/thumbnails/7.jpg)
7
• Collector implementations can either benon-concurrent or concurrent based on their characteristics
• This distinction is only relevant forparallel streams
• A non-concurrent collector can beused for either a sequential streamor a parallel stream!
Implementing Concurrent & Non-Concurrent Collectors
We’ll just focus on parallel streams in the subsequent discussion
![Page 8: Java 8 Parallel Stream Internals (Part 6)schmidt/cs891f/2018-PDFs/12-parallelstrea… · •return ArrayList::new Implementing Concurrent & Non-Concurrent Collectors A non-concurrent](https://reader030.vdocument.in/reader030/viewer/2022040309/5f0bdda47e708231d432991a/html5/thumbnails/8.jpg)
8
• A non-concurrent collector operates by merging sub-results
See stackoverflow.com/questions/22350288/parallel-streams-collectors-and-thread-safety
Implementing Concurrent & Non-Concurrent Collectors
![Page 9: Java 8 Parallel Stream Internals (Part 6)schmidt/cs891f/2018-PDFs/12-parallelstrea… · •return ArrayList::new Implementing Concurrent & Non-Concurrent Collectors A non-concurrent](https://reader030.vdocument.in/reader030/viewer/2022040309/5f0bdda47e708231d432991a/html5/thumbnails/9.jpg)
9
• A non-concurrent collector operates by merging sub-results
• The input source is partitioned into chunks
Implementing Concurrent & Non-Concurrent Collectors
join join
join
Processsequentially
Processsequentially
Processsequentially
Processsequentially
InputSource1.1 InputSource1.2 InputSource2.1 InputSource2.2
InputSource1 InputSource2
trySplit()
InputSource
trySplit() trySplit()
![Page 10: Java 8 Parallel Stream Internals (Part 6)schmidt/cs891f/2018-PDFs/12-parallelstrea… · •return ArrayList::new Implementing Concurrent & Non-Concurrent Collectors A non-concurrent](https://reader030.vdocument.in/reader030/viewer/2022040309/5f0bdda47e708231d432991a/html5/thumbnails/10.jpg)
10
• A non-concurrent collector operates by merging sub-results
• The input source is partitioned into chunks
• Each chunk runs in parallel & is collected into an intermediate mutable result container
• e.g., a list or a map
Implementing Concurrent & Non-Concurrent Collectors
join join
join
Processsequentially
Processsequentially
Processsequentially
Processsequentially
InputSource1.1 InputSource1.2 InputSource2.1 InputSource2.2
InputSource1 InputSource2
trySplit()
InputSource
trySplit() trySplit()
![Page 11: Java 8 Parallel Stream Internals (Part 6)schmidt/cs891f/2018-PDFs/12-parallelstrea… · •return ArrayList::new Implementing Concurrent & Non-Concurrent Collectors A non-concurrent](https://reader030.vdocument.in/reader030/viewer/2022040309/5f0bdda47e708231d432991a/html5/thumbnails/11.jpg)
11
• A non-concurrent collector operates by merging sub-results
• The input source is partitioned into chunks
• Each chunk runs in parallel & is collected into an intermediate mutable result container
• e.g., a list or a map
Implementing Concurrent & Non-Concurrent Collectors
join join
join
Processsequentially
Processsequentially
Processsequentially
Processsequentially
InputSource1.1 InputSource1.2 InputSource2.1 InputSource2.2
InputSource1 InputSource2
trySplit()
InputSource
trySplit() trySplit()
Different threads operate on different instances of intermediate result containers
![Page 12: Java 8 Parallel Stream Internals (Part 6)schmidt/cs891f/2018-PDFs/12-parallelstrea… · •return ArrayList::new Implementing Concurrent & Non-Concurrent Collectors A non-concurrent](https://reader030.vdocument.in/reader030/viewer/2022040309/5f0bdda47e708231d432991a/html5/thumbnails/12.jpg)
12
• A non-concurrent collector operates by merging sub-results
• The input source is partitioned into chunks
• Each chunk runs in parallel & is collected into an intermediate mutable result container
• These sub-results are then merged into a final mutable result container
• Only one thread in the fork-join pool is used to merge any pair of intermediate sub-results
Implementing Concurrent & Non-Concurrent Collectors
join join
join
Processsequentially
Processsequentially
Processsequentially
Processsequentially
InputSource1.1 InputSource1.2 InputSource2.1 InputSource2.2
InputSource1 InputSource2
trySplit()
InputSource
trySplit() trySplit()
![Page 13: Java 8 Parallel Stream Internals (Part 6)schmidt/cs891f/2018-PDFs/12-parallelstrea… · •return ArrayList::new Implementing Concurrent & Non-Concurrent Collectors A non-concurrent](https://reader030.vdocument.in/reader030/viewer/2022040309/5f0bdda47e708231d432991a/html5/thumbnails/13.jpg)
13
• A non-concurrent collector operates by merging sub-results
• The input source is partitioned into chunks
• Each chunk runs in parallel & is collected into an intermediate mutable result container
• These sub-results are then merged into a final mutable result container
• Only one thread in the fork-join pool is used to merge any pair of intermediate sub-results
Implementing Concurrent & Non-Concurrent Collectors
join join
join
Processsequentially
Processsequentially
Processsequentially
Processsequentially
InputSource1.1 InputSource1.2 InputSource2.1 InputSource2.2
InputSource1 InputSource2
trySplit()
InputSource
trySplit() trySplit()
Thus there’s no need for any synchronizers in a non-concurrent collector
![Page 14: Java 8 Parallel Stream Internals (Part 6)schmidt/cs891f/2018-PDFs/12-parallelstrea… · •return ArrayList::new Implementing Concurrent & Non-Concurrent Collectors A non-concurrent](https://reader030.vdocument.in/reader030/viewer/2022040309/5f0bdda47e708231d432991a/html5/thumbnails/14.jpg)
14
Processsequentially
Processsequentially
Processsequentially
Processsequentially
• A non-concurrent collector operates by merging sub-results
• The input source is partitioned into chunks
• Each chunk runs in parallel & is collected into an intermediate mutable result container
• These sub-results are then merged into a final mutable result container
Implementing Concurrent & Non-Concurrent Collectors
join join
joinThis process is safe & order-preserving, but costly for containers like maps & sets
![Page 15: Java 8 Parallel Stream Internals (Part 6)schmidt/cs891f/2018-PDFs/12-parallelstrea… · •return ArrayList::new Implementing Concurrent & Non-Concurrent Collectors A non-concurrent](https://reader030.vdocument.in/reader030/viewer/2022040309/5f0bdda47e708231d432991a/html5/thumbnails/15.jpg)
15
• A concurrent collector creates one concurrent mutable result container & accumulates elements into it from multiple threads in a parallel stream
See stackoverflow.com/questions/22350288/parallel-streams-collectors-and-thread-safety
Implementing Concurrent & Non-Concurrent Collectors
![Page 16: Java 8 Parallel Stream Internals (Part 6)schmidt/cs891f/2018-PDFs/12-parallelstrea… · •return ArrayList::new Implementing Concurrent & Non-Concurrent Collectors A non-concurrent](https://reader030.vdocument.in/reader030/viewer/2022040309/5f0bdda47e708231d432991a/html5/thumbnails/16.jpg)
16
• A concurrent collector creates one concurrent mutable result container & accumulates elements into it from multiple threads in a parallel stream
• As usual, the input source is partitioned into chunks
InputSource1.1 InputSource1.2 InputSource2.1 InputSource2.2
InputSource1 InputSource2
trySplit()
InputSource
trySplit() trySplit()
Implementing Concurrent & Non-Concurrent Collectors
![Page 17: Java 8 Parallel Stream Internals (Part 6)schmidt/cs891f/2018-PDFs/12-parallelstrea… · •return ArrayList::new Implementing Concurrent & Non-Concurrent Collectors A non-concurrent](https://reader030.vdocument.in/reader030/viewer/2022040309/5f0bdda47e708231d432991a/html5/thumbnails/17.jpg)
17
Processsequentially
Processsequentially
Processsequentially
Processsequentially
accumulate() accumulate()
accumulate()
• A concurrent collector creates one concurrent mutable result container & accumulates elements into it from multiple threads in a parallel stream
• As usual, the input source is partitioned into chunks
• Each chunk runs in parallel & is collected into one concurrent mutable result container
• e.g., a concurrent map or set
InputSource1.1 InputSource1.2 InputSource2.1 InputSource2.2
InputSource1 InputSource2
trySplit()
InputSource
trySplit() trySplit()
Implementing Concurrent & Non-Concurrent Collectors
ConcurrentResult Container
![Page 18: Java 8 Parallel Stream Internals (Part 6)schmidt/cs891f/2018-PDFs/12-parallelstrea… · •return ArrayList::new Implementing Concurrent & Non-Concurrent Collectors A non-concurrent](https://reader030.vdocument.in/reader030/viewer/2022040309/5f0bdda47e708231d432991a/html5/thumbnails/18.jpg)
18
Processsequentially
Processsequentially
Processsequentially
Processsequentially
accumulate() accumulate()
accumulate()
• A concurrent collector creates one concurrent mutable result container & accumulates elements into it from multiple threads in a parallel stream
• As usual, the input source is partitioned into chunks
• Each chunk runs in parallel & is collected into one concurrent mutable result container
• e.g., a concurrent map or set
InputSource1.1 InputSource1.2 InputSource2.1 InputSource2.2
InputSource1 InputSource2
trySplit()
InputSource
trySplit() trySplit()
Implementing Concurrent & Non-Concurrent Collectors
ConcurrentResult Container
Different threads in a parallel stream share one concurrent result container
![Page 19: Java 8 Parallel Stream Internals (Part 6)schmidt/cs891f/2018-PDFs/12-parallelstrea… · •return ArrayList::new Implementing Concurrent & Non-Concurrent Collectors A non-concurrent](https://reader030.vdocument.in/reader030/viewer/2022040309/5f0bdda47e708231d432991a/html5/thumbnails/19.jpg)
19
• A concurrent collector creates one concurrent mutable result container & accumulates elements into it from multiple threads in a parallel stream
• As usual, the input source is partitioned into chunks
• Each chunk runs in parallel & is collected into one concurrent mutable result container
join join
join
Processsequentially
Processsequentially
Processsequentially
Processsequentially
InputSource1.1 InputSource1.2 InputSource2.1 InputSource2.2
InputSource1 InputSource2
trySplit()
InputSource
trySplit() trySplit()
Of course, encounter order is not preserved..
Implementing Concurrent & Non-Concurrent Collectors
Thus there’s no need to merge any intermediate sub-results!
![Page 20: Java 8 Parallel Stream Internals (Part 6)schmidt/cs891f/2018-PDFs/12-parallelstrea… · •return ArrayList::new Implementing Concurrent & Non-Concurrent Collectors A non-concurrent](https://reader030.vdocument.in/reader030/viewer/2022040309/5f0bdda47e708231d432991a/html5/thumbnails/20.jpg)
20
• A concurrent collector may perform better than a non-concurrent collector if merging costs are high
Implementing Concurrent & Non-Concurrent Collectors
See github.com/douglascraigschmidt/LiveLessons/tree/master/Java8/ex14
![Page 21: Java 8 Parallel Stream Internals (Part 6)schmidt/cs891f/2018-PDFs/12-parallelstrea… · •return ArrayList::new Implementing Concurrent & Non-Concurrent Collectors A non-concurrent](https://reader030.vdocument.in/reader030/viewer/2022040309/5f0bdda47e708231d432991a/html5/thumbnails/21.jpg)
21
• A concurrent collector may perform better than a non-concurrent collector if merging costs are high
• Highly optimized result containers like ConcurrentHashMap may be more efficient than merging HashMaps
Implementing Concurrent & Non-Concurrent Collectors
See codepumpkin.com/hashtable-vs-synchronizedmap-vs-concurrenthashmap
![Page 22: Java 8 Parallel Stream Internals (Part 6)schmidt/cs891f/2018-PDFs/12-parallelstrea… · •return ArrayList::new Implementing Concurrent & Non-Concurrent Collectors A non-concurrent](https://reader030.vdocument.in/reader030/viewer/2022040309/5f0bdda47e708231d432991a/html5/thumbnails/22.jpg)
22
Implementing Concurrent & Non-Concurrent Collectors
See www.quora.com/What-is-the-difference-between-synchronize-and-concurrent-collection-in-Java
• A concurrent collector may perform better than a non-concurrent collector if merging costs are high
• Highly optimized result containers like ConcurrentHashMap may be more efficient than merging HashMaps
• ConcurrentHashMap is also moreefficient than a SynchronizedMap
![Page 23: Java 8 Parallel Stream Internals (Part 6)schmidt/cs891f/2018-PDFs/12-parallelstrea… · •return ArrayList::new Implementing Concurrent & Non-Concurrent Collectors A non-concurrent](https://reader030.vdocument.in/reader030/viewer/2022040309/5f0bdda47e708231d432991a/html5/thumbnails/23.jpg)
23See www.baeldung.com/java-8-collectors
Implementing Concurrent & Non-Concurrent Collectors• The Collector interface defines
three generic types
![Page 24: Java 8 Parallel Stream Internals (Part 6)schmidt/cs891f/2018-PDFs/12-parallelstrea… · •return ArrayList::new Implementing Concurrent & Non-Concurrent Collectors A non-concurrent](https://reader030.vdocument.in/reader030/viewer/2022040309/5f0bdda47e708231d432991a/html5/thumbnails/24.jpg)
24
Implementing Concurrent & Non-Concurrent Collectors• The Collector interface defines
three generic types
• T – The type of objects available
in the stream
• e.g., Integer, String, etc.
![Page 25: Java 8 Parallel Stream Internals (Part 6)schmidt/cs891f/2018-PDFs/12-parallelstrea… · •return ArrayList::new Implementing Concurrent & Non-Concurrent Collectors A non-concurrent](https://reader030.vdocument.in/reader030/viewer/2022040309/5f0bdda47e708231d432991a/html5/thumbnails/25.jpg)
25
Implementing Concurrent & Non-Concurrent Collectors• The Collector interface defines
three generic types
• T
• A – The type of a mutable accumulator object for collection
• e.g., ConcurrentHashSet orList of T (implemented byArrayList, LinkedList, etc.)
See Java8/ex14/src/main/java/utils/ConcurrentHashSet.java
![Page 26: Java 8 Parallel Stream Internals (Part 6)schmidt/cs891f/2018-PDFs/12-parallelstrea… · •return ArrayList::new Implementing Concurrent & Non-Concurrent Collectors A non-concurrent](https://reader030.vdocument.in/reader030/viewer/2022040309/5f0bdda47e708231d432991a/html5/thumbnails/26.jpg)
26See www.baeldung.com/java-8-collectors
Implementing Concurrent & Non-Concurrent Collectors• The Collector interface defines
three generic types
• T
• A
• R – The type of a final result
• e.g., ConcurrentHashSet orList of T
![Page 27: Java 8 Parallel Stream Internals (Part 6)schmidt/cs891f/2018-PDFs/12-parallelstrea… · •return ArrayList::new Implementing Concurrent & Non-Concurrent Collectors A non-concurrent](https://reader030.vdocument.in/reader030/viewer/2022040309/5f0bdda47e708231d432991a/html5/thumbnails/27.jpg)
27
• Five methods are defined in the Collector interface
Implementing Concurrent & Non-Concurrent Collectors
![Page 28: Java 8 Parallel Stream Internals (Part 6)schmidt/cs891f/2018-PDFs/12-parallelstrea… · •return ArrayList::new Implementing Concurrent & Non-Concurrent Collectors A non-concurrent](https://reader030.vdocument.in/reader030/viewer/2022040309/5f0bdda47e708231d432991a/html5/thumbnails/28.jpg)
28
• Five methods are defined in the Collector interface
• characteristics() – provides a
stream with additional information used for internal optimizations, e.g.
• UNORDERED
• The collector need not preservethe encounter order
Implementing Concurrent & Non-Concurrent Collectors
A concurrent collector should be UNORDERED, but a non-concurrent collector can be ORDERED
![Page 29: Java 8 Parallel Stream Internals (Part 6)schmidt/cs891f/2018-PDFs/12-parallelstrea… · •return ArrayList::new Implementing Concurrent & Non-Concurrent Collectors A non-concurrent](https://reader030.vdocument.in/reader030/viewer/2022040309/5f0bdda47e708231d432991a/html5/thumbnails/29.jpg)
29
• Five methods are defined in the Collector interface
• characteristics() – provides a
stream with additional information used for internal optimizations, e.g.
• UNORDERED
• IDENTIFY_FINISH
• The finisher() is the identity function so it can be a no-op
• e.g. finisher() just returns null
Implementing Concurrent & Non-Concurrent Collectors
A concurrent collector should be IDENTITY_FINISH, whereas a non-concurrent collector could be
![Page 30: Java 8 Parallel Stream Internals (Part 6)schmidt/cs891f/2018-PDFs/12-parallelstrea… · •return ArrayList::new Implementing Concurrent & Non-Concurrent Collectors A non-concurrent](https://reader030.vdocument.in/reader030/viewer/2022040309/5f0bdda47e708231d432991a/html5/thumbnails/30.jpg)
30
• Five methods are defined in the Collector interface
• characteristics() – provides a
stream with additional information used for internal optimizations, e.g.
• UNORDERED
• IDENTIFY_FINISH
• CONCURRENT
• accumulator() is called concurrently on result container
Implementing Concurrent & Non-Concurrent Collectors
A concurrent collector should be CONCURRENT, but a non-concurrent collector should not be!
The mutable result container must be synchronized!!
![Page 31: Java 8 Parallel Stream Internals (Part 6)schmidt/cs891f/2018-PDFs/12-parallelstrea… · •return ArrayList::new Implementing Concurrent & Non-Concurrent Collectors A non-concurrent](https://reader030.vdocument.in/reader030/viewer/2022040309/5f0bdda47e708231d432991a/html5/thumbnails/31.jpg)
31
• Five methods are defined in the Collector interface
• characteristics() – provides a
stream with additional information used for internal optimizations, e.g.
• UNORDERED
• IDENTIFY_FINISH
• CONCURRENT
• accumulator() is called concurrently on result container
• The combiner() method is a no-op
Implementing Concurrent & Non-Concurrent Collectors
![Page 32: Java 8 Parallel Stream Internals (Part 6)schmidt/cs891f/2018-PDFs/12-parallelstrea… · •return ArrayList::new Implementing Concurrent & Non-Concurrent Collectors A non-concurrent](https://reader030.vdocument.in/reader030/viewer/2022040309/5f0bdda47e708231d432991a/html5/thumbnails/32.jpg)
32
• Five methods are defined in the Collector interface
• characteristics() – provides a
stream with additional information used for internal optimizations, e.g.
• UNORDERED
• IDENTIFY_FINISH
• CONCURRENT
• accumulator() is called concurrently on result container
• The combiner() method is a no-op
• A non-concurrent collector can be used with either sequential or parallel streams
Implementing Concurrent & Non-Concurrent Collectors
Internally, the streams framework decides how to ensure correct behavior
![Page 33: Java 8 Parallel Stream Internals (Part 6)schmidt/cs891f/2018-PDFs/12-parallelstrea… · •return ArrayList::new Implementing Concurrent & Non-Concurrent Collectors A non-concurrent](https://reader030.vdocument.in/reader030/viewer/2022040309/5f0bdda47e708231d432991a/html5/thumbnails/33.jpg)
33
Set characteristics() {
return Collections.unmodifiableSet
(EnumSet.of(Collector.Characteristics.CONCURRENT,
Collector.Characteristics.UNORDERED,
Collector.Characteristics.IDENTITY_FINISH));
}
• Five methods are defined in the Collector interface
• characteristics() – provides a
stream with additional information used for internal optimizations, e.g.
Implementing Concurrent & Non-Concurrent Collectors
See docs.oracle.com/javase/8/docs/api/java/util/EnumSet.html
Any/all characteristics can be set using EnumSet.of()
![Page 34: Java 8 Parallel Stream Internals (Part 6)schmidt/cs891f/2018-PDFs/12-parallelstrea… · •return ArrayList::new Implementing Concurrent & Non-Concurrent Collectors A non-concurrent](https://reader030.vdocument.in/reader030/viewer/2022040309/5f0bdda47e708231d432991a/html5/thumbnails/34.jpg)
34
• Five methods are defined in the Collector interface
• characteristics()
• supplier() – returns a supplier
that acts as a factory to generate an empty result container
Implementing Concurrent & Non-Concurrent Collectors
![Page 35: Java 8 Parallel Stream Internals (Part 6)schmidt/cs891f/2018-PDFs/12-parallelstrea… · •return ArrayList::new Implementing Concurrent & Non-Concurrent Collectors A non-concurrent](https://reader030.vdocument.in/reader030/viewer/2022040309/5f0bdda47e708231d432991a/html5/thumbnails/35.jpg)
35
• Five methods are defined in the Collector interface
• characteristics()
• supplier() – returns a supplier
that acts as a factory to generate an empty result container, e.g.
• return ArrayList::new
Implementing Concurrent & Non-Concurrent Collectors
A non-concurrent collector provides a result container for each thread in a parallel stream
![Page 36: Java 8 Parallel Stream Internals (Part 6)schmidt/cs891f/2018-PDFs/12-parallelstrea… · •return ArrayList::new Implementing Concurrent & Non-Concurrent Collectors A non-concurrent](https://reader030.vdocument.in/reader030/viewer/2022040309/5f0bdda47e708231d432991a/html5/thumbnails/36.jpg)
36
• Five methods are defined in the Collector interface
• characteristics()
• supplier() – a factory that returns a
supplier instance that generates an empty result container, e.g.
• return ArrayList::new
• return ConcurrentHashSet::new
Implementing Concurrent & Non-Concurrent Collectors
A concurrent collector has one result container shared by all threads in a parallel stream
![Page 37: Java 8 Parallel Stream Internals (Part 6)schmidt/cs891f/2018-PDFs/12-parallelstrea… · •return ArrayList::new Implementing Concurrent & Non-Concurrent Collectors A non-concurrent](https://reader030.vdocument.in/reader030/viewer/2022040309/5f0bdda47e708231d432991a/html5/thumbnails/37.jpg)
37
• Five methods are defined in the Collector interface
• characteristics()
• supplier()
• accumulator() – returns a bi-
consumer that adds a new element to an existing result container
Implementing Concurrent & Non-Concurrent Collectors
![Page 38: Java 8 Parallel Stream Internals (Part 6)schmidt/cs891f/2018-PDFs/12-parallelstrea… · •return ArrayList::new Implementing Concurrent & Non-Concurrent Collectors A non-concurrent](https://reader030.vdocument.in/reader030/viewer/2022040309/5f0bdda47e708231d432991a/html5/thumbnails/38.jpg)
38
• Five methods are defined in the Collector interface
• characteristics()
• supplier()
• accumulator() – returns a bi-
consumer that adds a new element to an existing result container, e.g.
• return List::add
Implementing Concurrent & Non-Concurrent Collectors
A non-concurrent collector needs no synchronization
![Page 39: Java 8 Parallel Stream Internals (Part 6)schmidt/cs891f/2018-PDFs/12-parallelstrea… · •return ArrayList::new Implementing Concurrent & Non-Concurrent Collectors A non-concurrent](https://reader030.vdocument.in/reader030/viewer/2022040309/5f0bdda47e708231d432991a/html5/thumbnails/39.jpg)
39
• Five methods are defined in the Collector interface
• characteristics()
• supplier()
• accumulator() – returns a
bi-consumer that adds a new element to result container, e.g.
• return List::add
• return ConcurrentHashSet::add
Implementing Concurrent & Non-Concurrent Collectors
A non-concurrent collector must be synchronized
![Page 40: Java 8 Parallel Stream Internals (Part 6)schmidt/cs891f/2018-PDFs/12-parallelstrea… · •return ArrayList::new Implementing Concurrent & Non-Concurrent Collectors A non-concurrent](https://reader030.vdocument.in/reader030/viewer/2022040309/5f0bdda47e708231d432991a/html5/thumbnails/40.jpg)
40
• Five methods are defined in the Collector interface
• characteristics()
• supplier()
• accumulator()
• combiner() – returns a binary
operator that merges two result containers together
Implementing Concurrent & Non-Concurrent Collectors
![Page 41: Java 8 Parallel Stream Internals (Part 6)schmidt/cs891f/2018-PDFs/12-parallelstrea… · •return ArrayList::new Implementing Concurrent & Non-Concurrent Collectors A non-concurrent](https://reader030.vdocument.in/reader030/viewer/2022040309/5f0bdda47e708231d432991a/html5/thumbnails/41.jpg)
41
• Five methods are defined in the Collector interface
• characteristics()
• supplier()
• accumulator()
• combiner() – returns a binary
operator that merges two result containers together, e.g.
• return (one, another) -> {
one.addAll(another); return one;
}
Implementing Concurrent & Non-Concurrent Collectors
A combiner() is only used for a non-concurrent collector
![Page 42: Java 8 Parallel Stream Internals (Part 6)schmidt/cs891f/2018-PDFs/12-parallelstrea… · •return ArrayList::new Implementing Concurrent & Non-Concurrent Collectors A non-concurrent](https://reader030.vdocument.in/reader030/viewer/2022040309/5f0bdda47e708231d432991a/html5/thumbnails/42.jpg)
42
• Five methods are defined in the Collector interface
• characteristics()
• supplier()
• accumulator()
• combiner() – returns a binary
operator that merges two result containers together, e.g.
• return (one, another) -> {
one.addAll(another); return one;
}
• return null
Implementing Concurrent & Non-Concurrent Collectors
The combiner() method is not called when CONCURRENT is set
![Page 43: Java 8 Parallel Stream Internals (Part 6)schmidt/cs891f/2018-PDFs/12-parallelstrea… · •return ArrayList::new Implementing Concurrent & Non-Concurrent Collectors A non-concurrent](https://reader030.vdocument.in/reader030/viewer/2022040309/5f0bdda47e708231d432991a/html5/thumbnails/43.jpg)
43
• Five methods are defined in the Collector interface
• characteristics()
• supplier()
• accumulator()
• combiner()
• finisher() – returns a function
that converts the result containerto final result type
Implementing Concurrent & Non-Concurrent Collectors
![Page 44: Java 8 Parallel Stream Internals (Part 6)schmidt/cs891f/2018-PDFs/12-parallelstrea… · •return ArrayList::new Implementing Concurrent & Non-Concurrent Collectors A non-concurrent](https://reader030.vdocument.in/reader030/viewer/2022040309/5f0bdda47e708231d432991a/html5/thumbnails/44.jpg)
44
• Five methods are defined in the Collector interface
• characteristics()
• supplier()
• accumulator()
• combiner()
• finisher() – returns a function
that converts the result containerto final result type, e.g.
• Function.identity()
Implementing Concurrent & Non-Concurrent Collectors
![Page 45: Java 8 Parallel Stream Internals (Part 6)schmidt/cs891f/2018-PDFs/12-parallelstrea… · •return ArrayList::new Implementing Concurrent & Non-Concurrent Collectors A non-concurrent](https://reader030.vdocument.in/reader030/viewer/2022040309/5f0bdda47e708231d432991a/html5/thumbnails/45.jpg)
45
• Five methods are defined in the Collector interface
• characteristics()
• supplier()
• accumulator()
• combiner()
• finisher() – returns a function
that converts the result containerto final result type, e.g.
• Function.identity()
• return null
Implementing Concurrent & Non-Concurrent Collectors
Should be a no-op if IDENTITY_FINISH characteristic is set
![Page 46: Java 8 Parallel Stream Internals (Part 6)schmidt/cs891f/2018-PDFs/12-parallelstrea… · •return ArrayList::new Implementing Concurrent & Non-Concurrent Collectors A non-concurrent](https://reader030.vdocument.in/reader030/viewer/2022040309/5f0bdda47e708231d432991a/html5/thumbnails/46.jpg)
46
• Five methods are defined in the Collector interface
• characteristics()
• supplier()
• accumulator()
• combiner()
• finisher() – returns a function
that converts the result containerto final result type, e.g.
• Function.identity()
• return null
Implementing Concurrent & Non-Concurrent Collectors
See Java8/ex19/src/main/java/utils/FuturesCollector.java
Stream
.generate(() ->
makeBigFraction
(new Random(), false))
.limit(sMAX_FRACTIONS)
.map(reduceAndMultiplyFraction)
.collect(FuturesCollector
.toFuture())
.thenAccept
(this::sortAndPrintList);
finisher() can also be much more interesting!
![Page 47: Java 8 Parallel Stream Internals (Part 6)schmidt/cs891f/2018-PDFs/12-parallelstrea… · •return ArrayList::new Implementing Concurrent & Non-Concurrent Collectors A non-concurrent](https://reader030.vdocument.in/reader030/viewer/2022040309/5f0bdda47e708231d432991a/html5/thumbnails/47.jpg)
47
End of Java 8 Parallel Stream Internals (Part 6)