java 8 completablefutures imagestreamgang example (part 2)schmidt/cs891f/2018-pdfs/...asynchronously...

Java 8 CompletableFutures

ImageStreamGang Example (Part 2)

Douglas C. Schmidtd.schmidt@vanderbilt.edu

www.dre.vanderbilt.edu/~schmidt

Professor of Computer Science

Institute for Software

Integrated Systems

Vanderbilt University

Nashville, Tennessee, USA

Learning Objectives in this Part of the Lesson• Understand the design of the

Java 8 completable futureversion of ImageStreamGang

• Know how to apply completablefutures to ImageStreamGang

SocketSocket

List of URLs to Download

List of Filters to Apply

Persistent

Data Store

Learning Objectives in this Part of the Lesson• Understand the design of the

Java 8 completable futureversion of ImageStreamGang

• Know how to apply completablefutures to ImageStreamGang, e.g.

• Factory methods & completionstage methods

Applying CompletableFutures to ImageStreamGang

• Focus on processStream()

Applying Completable Futures to ImageStreamGangvoid processStream() {

List<URL> urls = getInput();

CompletableFuture<Stream<Image>>

resultsFuture = urls

.stream()

.map(this::checkUrlCachedAsync)

.map(this::downloadImageAsync)

.flatMap(this::applyFiltersAsync)

.collect(toFuture())

.thenApply(stream ->

log(stream.flatMap

(Optional::stream),

urls.size()))

.join();

...See imagestreamgang/streams/ImageStreamCompletableFuture1.java

.stream()

log(stream.flatMap

(Optional::stream),

urls.size()))

.join();

Get the list of URLs input by the user

.stream()

log(stream.flatMap

(Optional::stream),

urls.size()))

.join();

Combines a Java 8 sequential stream with

completable futures

.stream()

log(stream.flatMap

(Optional::stream),

urls.size()))

.join();

Factory method creates a stream of URLs

• This implementation is verydifferent from parallel streams

.stream()

log(stream.flatMap

(Optional::stream),

urls.size()))

.join();

Asynchronously check if images are already cached locally

map() converts a stream of URLs to a stream of futures to optional URLs

.stream()

log(stream.flatMap

(Optional::stream),

urls.size()))

.join();

Asynchronously download an image at each given URL

map() converts URL futures (completed) to image futures (downloading)

.stream()

log(stream.flatMap

(Optional::stream),

urls.size()))

.join();

Asynchronously filter & store downloaded images on the local file system

flatMap() converts image futures (completed) to filtered image futures (xforming/storing)

.stream()

log(stream.flatMap

(Optional::stream),

urls.size()))

.join();

Create a future used to wait for all async operations associated w/the

stream of futures to complete

See next part on “arbitrary-arity” methods in CompletableFuture

.stream()

log(stream.flatMap

(Optional::stream),

urls.size()))

.join();

This lambda logs the results when all the futures in stream

complete their async processing

.stream()

log(stream.flatMap

(Optional::stream),

urls.size()))

.join();

Wait until all the images have been downloaded,

processed, & stored

This is the one & only call to join() in this implementation strategy

Applying Factory Methods in ImageStreamGang

• Initiate an async check to see if images are cached locally

Applying Factory Methods in ImageStreamGangvoid processStream() {

.stream()

log(stream.flatMap

(Optional::stream),

urls.size()))

.join();

map() calls the behavior checkUrlCachedAsync()

.stream()

log(stream.flatMap

(Optional::stream),

urls.size()))

.join();

Asynchronously check if a URL is already downloaded

.stream()

log(stream.flatMap

(Optional::stream),

urls.size()))

.join();

Returns a stream of completablefutures to optional URLs, which have a value if the URL is not

cached or are empty if it is cached

Later behaviors simply ignore “empty” optional URL values

• checkUrlCachedAsync() uses the supplyAsync() factory method internally

See imagestreamgang/streams/ImageStreamCompletableFutureBase.java

CompletableFuture<Optional<URL>> checkUrlCachedAsync(URL url) {

return CompletableFuture

.supplyAsync(() ->

Optional.ofNullable(urlCached(url)

? null

: url),

getExecutor());

.supplyAsync(() ->

? null

: url),

getExecutor());

This factory method registers an action that runs asynchronously

See docs.oracle.com/javase/8/docs/api/java/util/concurrent/CompletableFuture.html#supplyAsync

void initiateStream() {

// Set the executor to the common fork-join pool.

setExecutor(ForkJoinPool.commonPool());

.supplyAsync(() ->

? null

: url),

getExecutor());

supplyAsync() runs action in a worker thread from the common fork-join pool

See dzone.com/articles/common-fork-join-pool-and-streams

.supplyAsync(() ->

? null

: url),

getExecutor());

ofNullable() is a factory method that returns a completable future to an optional URL, which has a value if the URL is not cached or is empty if it is cached

See docs.oracle.com/javase/8/docs/api/java/util/Optional.html#ofNullable

See imagestreamgang/streams/ImageStreamGang.java

boolean urlCached(URL url) {

return mFilters.stream()

.filter(filter -> urlCached(url, filter.getName()))

.count() > 0;

.supplyAsync(() ->

? null

: url),

getExecutor());

Returns true if the image has already been filtered before

.supplyAsync(() ->

? null

: url),

getExecutor());

boolean urlCached(URL url, String filterName) {

File file = new File(getPath(), filterName);

File imageFile = new File(file, getNameForUrl(url));

return !imageFile.createNewFile();

Returns true if image file already exists

.supplyAsync(() ->

? null

: url),

getExecutor());

There are clearly better ways of implementing an image cache!

boolean urlCached(URL url, String filterName) {

File file = new File(getPath(), filterName);

File imageFile = new File(file, getNameForUrl(url));

return !imageFile.createNewFile();

Applying Completion Stage Methods in ImageStreamGang

• Asynchronously download an image at each given URL

Applying Completion Stage Methods in ImageStreamGangvoid processStream() {

.stream()

log(stream.flatMap

(Optional::stream),

urls.size()))

.join();

map() calls the behaviordownloadImageAsync()

Asynchronously downloads an image & stores it in memory

void processStream() {

.stream()

log(stream.flatMap

(Optional::stream),

urls.size()))

.join();

Later behaviors simply ignore “empty” optional images

Returns a stream of futures to optional images, which have a value if the image is being downloaded or

are empty if it is already cached

.stream()

log(stream.flatMap

(Optional::stream),

urls.size()))

.join();

• downloadImageAsync() uses the thenApplyAsync() method internally

CompletableFuture<Optional<Image>> downloadImageAsync

(CompletableFuture<Optional<URL>> urlFuture) {

return urlFuture

.thenApplyAsync(urlOpt ->

urlOpt

.map(this::blockingDownload),

getExecutor());

Asynchronously download image when future completes

See imagestreamgang/streams/ImageStreamCompletableFuture1.java

Applying Completion Stage Methods in ImageStreamGang• downloadImageAsync() uses the thenApplyAsync() method internally

return urlFuture

urlOpt

getExecutor());

This completion stage method registers an action that’s not executed immediately, but only after the future completes

See docs.oracle.com/javase/8/docs/api/java/util/concurrent/CompletableFuture.html#thenApplyAsync

return urlFuture

urlOpt

getExecutor());

If a url is present when the future completes download it & return an optional describing the result; otherwise return an empty optional

See docs.oracle.com/javase/8/docs/api/java/util/Optional.html#map

Asynchronously run blockingDownload() if the url is non-empty

return urlFuture

urlOpt

getExecutor());

34You could also use a ThreadPoolExecutor (fixed-size or cached)

return urlFuture

urlOpt

getExecutor());

Use the common fork-join pool & its ManagedBlocker mechanism

Image blockingDownload(URL url) {

return BlockingTask

.callInManagedBlock(() ->

downloadImage(url));

Transform a URL into an Image by downloading each image via the URL

Image blockingDownload(URL url) {

return BlockingTask

.callInManagedBlock(() ->

downloadImage(url));

BlockingTask.callInManagedBlock() wraps the ManagedBlocker interface, which expands the common fork/join thread pool to handle the blocking image download

See imagestreamgang/utils/BlockingTask.java

Returns a future to an image that completes when the image is finished downloading

return urlFuture

urlOpt

getExecutor());

• Asynchronously filter & store downloaded images on the local file system

.stream()

log(stream.flatMap

(Optional::stream),

urls.size()))

.join();

flatMap() calls behavior applyFiltersAsync()

Asynchronous filter images & store them into files

.stream()

log(stream.flatMap

(Optional::stream),

urls.size()))

.join();

Later operations ignore “empty” optional images

“Flatten” all filtered/stored images into a single output stream

.stream()

log(stream.flatMap

(Optional::stream),

urls.size()))

.join();

See docs.oracle.com/javase/8/docs/api/java/util/stream/Stream.html#flatMap

.stream()

log(stream.flatMap

(Optional::stream),

urls.size()))

.join();

Returns a stream of futures to optional images, which have a

value if the image is being filtered or are empty if it is already cached

• applyFiltersAsync() uses the thenApplyAsync() method internally

See imagestreamgang/streams/ImageStreamCompletableFuture1.java

Stream<CompletableFuture<Optional<Image>>> applyFiltersAsync

(CompletableFuture<Optional<Image>> imageFuture) {

return mFilters

.stream()

.map(filter -> imageFuture

.thenApplyAsync(imageOpt ->

imageOpt

.map(image ->

makeFilterDecoratorWithImage

(filter, image).run()),

getExecutor()));

Asynchronously filter images & then store them into files

Applying Completion Stage Methods in ImageStreamGang• applyFiltersAsync() uses the thenApplyAsync() method internally

return mFilters

.stream()

imageOpt

.map(image ->

getExecutor()));

Convert the list of filters into a stream

return mFilters

.stream()

imageOpt

.map(image ->

getExecutor()));

Asynchronously apply a filter action after the previous stage completes

See docs.oracle.com/javase/8/docs/api/java/util/concurrent/CompletableFuture.html#thenApplyAsync

return mFilters

.stream()

imageOpt

.map(image ->

getExecutor()));

This completion stage method registers an action that’s not executed immediately, but runs only after the future completes

return mFilters

.stream()

imageOpt

.map(image ->

getExecutor()));}

If an image is present then perform the action & return optional containing result;

otherwise return an empty optional

See docs.oracle.com/javase/8/docs/api/java/util/Optional.html#map

return mFilters

.stream()

imageOpt

.map(image ->

getExecutor()));}

If an image is non-null then asynchronously filter the image & store it in an output file

return mFilters

.stream()

imageOpt

.map(image ->

getExecutor()));}

thenApplyAsync() runs the actions in a thread from the common fork-join pool

return mFilters

.stream()

imageOpt

.map(image ->

getExecutor()));

It also returns a new completable future that will trigger when the image has been filtered/stored

return mFilters

.stream()

imageOpt

.map(image ->

getExecutor()));

applyFiltersAsync() returns a stream of completable

futures to optional images

.stream()

log(stream.flatMap

(Optional::stream),

urls.size()))

.join();

flatMap() merges the stream of futures returned by applyFilters

Async() into a single stream

This stream is processed by collect(), as discussed in the next part of the lesson

End of Java 8 CompletableFutures ImageStreamGang

Example (Part 2)

java 8 completablefutures imagestreamgang example (part 2)schmidt/cs891f/2018-pdfs/...asynchronously...

Documents

concepts and technologies for distributed systems and big...

map, flatmap and reduce are your new best friends: simpler...

flatmap-front-17x11-012219...red train tours river gori...

java 8 completablefutures imagestreamgang example (part...

flatmap-17x11-071317 · 2017. 8. 2. · fountain of youth...

flatmap oslo presentation slides

java 8 completablefutures imagestreamgang example (part...

scala collection methods flatmap and flatten are more...

functional programming in java 8 - workshop at flatmap oslo...

effect systems in scala: beyond flatmap

free monads - functional vilniusthe problem with free monads...

asynchronous programming using completablefutures in java

a friendly critique of sparkr - hp labs · intro to spark...

map(), flatmap() and reduce() are your new best friends:...

by mario fusco mario.fusco@gmail.com twitter:...

java 8 parallel imagestreamgang example (part...

map, flatmap and reduce are your new best friends (javaone,...

the future of futures - a talk about java 8...

package ‘etas’ - r · flatmap=true, dist.unit =...

bm4 flatmap 4 beta 3.1 (rat) - larry...