java 8 completablefutures imagestreamgang example (part 2)schmidt/cs891f/2018-pdfs/...asynchronously...

Java 8 CompletableFutures

ImageStreamGang Example (Part 2)

Douglas C. [email protected]

www.dre.vanderbilt.edu/~schmidt

Professor of Computer Science

Institute for Software

Integrated Systems

Vanderbilt University

Nashville, Tennessee, USA

mailto:[email protected]

2

Learning Objectives in this Part of the Lesson• Understand the design of the

Java 8 completable futureversion of ImageStreamGang

• Know how to apply completablefutures to ImageStreamGang

SocketSocket

List of URLs to Download

…

List of Filters to Apply

Persistent

Data Store

3

Learning Objectives in this Part of the Lesson• Understand the design of the

Java 8 completable futureversion of ImageStreamGang

• Know how to apply completablefutures to ImageStreamGang, e.g.

• Factory methods & completionstage methods

4

Applying CompletableFutures to ImageStreamGang

5

• Focus on processStream()

Applying Completable Futures to ImageStreamGangvoid processStream() {

List<URL> urls = getInput();

CompletableFuture<Stream<Image>>

resultsFuture = urls

.stream()

.map(this::checkUrlCachedAsync)

.map(this::downloadImageAsync)

.flatMap(this::applyFiltersAsync)

.collect(toFuture())

.thenApply(stream ->

log(stream.flatMap

(Optional::stream),

urls.size()))

.join();

...See imagestreamgang/streams/ImageStreamCompletableFuture1.java

https://github.com/douglascraigschmidt/LiveLessons/blob/master/ImageStreamGang/AndroidGUI/app/src/main/java/livelessons/imagestreamgang/streams/ImageStreamCompletableFuture1.java

6






.stream()






log(stream.flatMap

(Optional::stream),

urls.size()))

.join();

Get the list of URLs input by the user

7






.stream()






log(stream.flatMap

(Optional::stream),

urls.size()))

.join();

Combines a Java 8 sequential stream with

completable futures

8






.stream()






log(stream.flatMap

(Optional::stream),

urls.size()))

.join();

Factory method creates a stream of URLs

9


• This implementation is verydifferent from parallel streams





.stream()






log(stream.flatMap

(Optional::stream),

urls.size()))

.join();

Asynchronously check if images are already cached locally

map() converts a stream of URLs to a stream of futures to optional URLs

10







.stream()






log(stream.flatMap

(Optional::stream),

urls.size()))

.join();

Asynchronously download an image at each given URL

map() converts URL futures (completed) to image futures (downloading)

11







.stream()






log(stream.flatMap

(Optional::stream),

urls.size()))

.join();

Asynchronously filter & store downloaded images on the local file system

flatMap() converts image futures (completed) to filtered image futures (xforming/storing)

12







.stream()






log(stream.flatMap

(Optional::stream),

urls.size()))

.join();

Create a future used to wait for all async operations associated w/the

stream of futures to complete

See next part on “arbitrary-arity” methods in CompletableFuture

13







.stream()






log(stream.flatMap

(Optional::stream),

urls.size()))

.join();

This lambda logs the results when all the futures in stream

complete their async processing

14







.stream()






log(stream.flatMap

(Optional::stream),

urls.size()))

.join();

Wait until all the images have been downloaded,

processed, & stored

This is the one & only call to join() in this implementation strategy

15

Applying Factory Methods in ImageStreamGang

16

• Initiate an async check to see if images are cached locally

Applying Factory Methods in ImageStreamGangvoid processStream() {




.stream()






log(stream.flatMap

(Optional::stream),

urls.size()))

.join();

map() calls the behavior checkUrlCachedAsync()

17






.stream()






log(stream.flatMap

(Optional::stream),

urls.size()))

.join();

Asynchronously check if a URL is already downloaded

18






.stream()






log(stream.flatMap

(Optional::stream),

urls.size()))

.join();

Returns a stream of completablefutures to optional URLs, which have a value if the URL is not

cached or are empty if it is cached

Later behaviors simply ignore “empty” optional URL values

19

• checkUrlCachedAsync() uses the supplyAsync() factory method internally


See imagestreamgang/streams/ImageStreamCompletableFutureBase.java

CompletableFuture<Optional<URL>> checkUrlCachedAsync(URL url) {

return CompletableFuture

.supplyAsync(() ->

Optional.ofNullable(urlCached(url)

? null

: url),

getExecutor());

}

https://github.com/douglascraigschmidt/LiveLessons/blob/master/ImageStreamGang/AndroidGUI/app/src/main/java/livelessons/imagestreamgang/streams/ImageStreamCompletableFutureBase.java

20





.supplyAsync(() ->


? null

: url),

getExecutor());

}

This factory method registers an action that runs asynchronously

See docs.oracle.com/javase/8/docs/api/java/util/concurrent/CompletableFuture.html#supplyAsync

https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/CompletableFuture.html#supplyAsync-java.util.function.Supplier-

21

void initiateStream() {

// Set the executor to the common fork-join pool.

setExecutor(ForkJoinPool.commonPool());

...

}





.supplyAsync(() ->


? null

: url),

getExecutor());

}

supplyAsync() runs action in a worker thread from the common fork-join pool

See dzone.com/articles/common-fork-join-pool-and-streams

https://dzone.com/articles/common-fork-join-pool-and-streams

22





.supplyAsync(() ->


? null

: url),

getExecutor());

}

ofNullable() is a factory method that returns a completable future to an optional URL, which has a value if the URL is not cached or is empty if it is cached

See docs.oracle.com/javase/8/docs/api/java/util/Optional.html#ofNullable

https://docs.oracle.com/javase/8/docs/api/java/util/Optional.html#ofNullable-T-

23



See imagestreamgang/streams/ImageStreamGang.java

boolean urlCached(URL url) {

return mFilters.stream()

.filter(filter -> urlCached(url, filter.getName()))

.count() > 0;

}



.supplyAsync(() ->


? null

: url),

getExecutor());

}

Returns true if the image has already been filtered before

https://github.com/douglascraigschmidt/LiveLessons/blob/master/ImageStreamGang/AndroidGUI/app/src/main/java/livelessons/imagestreamgang/streams/ImageStreamGang.java

24






.supplyAsync(() ->


? null

: url),

getExecutor());

}

boolean urlCached(URL url, String filterName) {

File file = new File(getPath(), filterName);

File imageFile = new File(file, getNameForUrl(url));

return !imageFile.createNewFile();

}

Returns true if image file already exists


25





.supplyAsync(() ->


? null

: url),

getExecutor());

}

There are clearly better ways of implementing an image cache!

boolean urlCached(URL url, String filterName) {

File file = new File(getPath(), filterName);

File imageFile = new File(file, getNameForUrl(url));

return !imageFile.createNewFile();

}

26

Applying Completion Stage Methods in ImageStreamGang

27

• Asynchronously download an image at each given URL

Applying Completion Stage Methods in ImageStreamGangvoid processStream() {




.stream()






log(stream.flatMap

(Optional::stream),

urls.size()))

.join();

map() calls the behaviordownloadImageAsync()

28



Asynchronously downloads an image & stores it in memory

void processStream() {




.stream()






log(stream.flatMap

(Optional::stream),

urls.size()))

.join();

Later behaviors simply ignore “empty” optional images

29



Returns a stream of futures to optional images, which have a value if the image is being downloaded or

are empty if it is already cached





.stream()






log(stream.flatMap

(Optional::stream),

urls.size()))

.join();

30

• downloadImageAsync() uses the thenApplyAsync() method internally

CompletableFuture<Optional<Image>> downloadImageAsync

(CompletableFuture<Optional<URL>> urlFuture) {

return urlFuture

.thenApplyAsync(urlOpt ->

urlOpt

.map(this::blockingDownload),

getExecutor());

}

Asynchronously download image when future completes


See imagestreamgang/streams/ImageStreamCompletableFuture1.java

https://github.com/douglascraigschmidt/LiveLessons/blob/master/ImageStreamGang/AndroidGUI/app/src/main/java/livelessons/imagestreamgang/streams/ImageStreamCompletableFuture1.java

31

Applying Completion Stage Methods in ImageStreamGang• downloadImageAsync() uses the thenApplyAsync() method internally



return urlFuture


urlOpt


getExecutor());

}

This completion stage method registers an action that’s not executed immediately, but only after the future completes

See docs.oracle.com/javase/8/docs/api/java/util/concurrent/CompletableFuture.html#thenApplyAsync

https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/CompletableFuture.html#thenApplyAsync-java.util.function.Function-java.util.concurrent.Executor-

32




return urlFuture


urlOpt


getExecutor());

}

If a url is present when the future completes download it & return an optional describing the result; otherwise return an empty optional

See docs.oracle.com/javase/8/docs/api/java/util/Optional.html#map

https://docs.oracle.com/javase/8/docs/api/java/util/Optional.html#map-java.util.function.Function-

33

Asynchronously run blockingDownload() if the url is non-empty




return urlFuture


urlOpt


getExecutor());

}

34You could also use a ThreadPoolExecutor (fixed-size or cached)




return urlFuture


urlOpt


getExecutor());

}


Use the common fork-join pool & its ManagedBlocker mechanism

35


Image blockingDownload(URL url) {

return BlockingTask

.callInManagedBlock(() ->

downloadImage(url));

}



Transform a URL into an Image by downloading each image via the URL


36


Image blockingDownload(URL url) {

return BlockingTask

.callInManagedBlock(() ->

downloadImage(url));

}


BlockingTask.callInManagedBlock() wraps the ManagedBlocker interface, which expands the common fork/join thread pool to handle the blocking image download

See imagestreamgang/utils/BlockingTask.java

https://github.com/douglascraigschmidt/LiveLessons/blob/master/ImageStreamGang/AndroidGUI/app/src/main/java/livelessons/imagestreamgang/utils/BlockingTask.java

37

Returns a future to an image that completes when the image is finished downloading




return urlFuture


urlOpt


getExecutor());

}


38

• Asynchronously filter & store downloaded images on the local file system





.stream()






log(stream.flatMap

(Optional::stream),

urls.size()))

.join();

flatMap() calls behavior applyFiltersAsync()

39



Asynchronous filter images & store them into files





.stream()






log(stream.flatMap

(Optional::stream),

urls.size()))

.join();

Later operations ignore “empty” optional images

40



“Flatten” all filtered/stored images into a single output stream





.stream()






log(stream.flatMap

(Optional::stream),

urls.size()))

.join();

See docs.oracle.com/javase/8/docs/api/java/util/stream/Stream.html#flatMap

https://docs.oracle.com/javase/8/docs/api/java/util/stream/Stream.html#flatMap-java.util.function.Function-

41






.stream()






log(stream.flatMap

(Optional::stream),

urls.size()))

.join();

Returns a stream of futures to optional images, which have a

value if the image is being filtered or are empty if it is already cached

42

• applyFiltersAsync() uses the thenApplyAsync() method internally

See imagestreamgang/streams/ImageStreamCompletableFuture1.java


Stream<CompletableFuture<Optional<Image>>> applyFiltersAsync

(CompletableFuture<Optional<Image>> imageFuture) {

return mFilters

.stream()

.map(filter -> imageFuture

.thenApplyAsync(imageOpt ->

imageOpt

.map(image ->

makeFilterDecoratorWithImage

(filter, image).run()),

getExecutor()));

}

Asynchronously filter images & then store them into files

https://github.com/douglascraigschmidt/LiveLessons/blob/master/ImageStreamGang/AndroidGUI/app/src/main/java/livelessons/imagestreamgang/streams/ImageStreamCompletableFutureBase.java

43

Applying Completion Stage Methods in ImageStreamGang• applyFiltersAsync() uses the thenApplyAsync() method internally



return mFilters

.stream()



imageOpt

.map(image ->



getExecutor()));

}

Convert the list of filters into a stream

44




return mFilters

.stream()



imageOpt

.map(image ->



getExecutor()));

}


Asynchronously apply a filter action after the previous stage completes

See docs.oracle.com/javase/8/docs/api/java/util/concurrent/CompletableFuture.html#thenApplyAsync

https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/CompletableFuture.html#thenApplyAsync-java.util.function.Function-java.util.concurrent.Executor-

45




return mFilters

.stream()



imageOpt

.map(image ->



getExecutor()));

}


This completion stage method registers an action that’s not executed immediately, but runs only after the future completes

46



return mFilters

.stream()



imageOpt

.map(image ->



getExecutor()));}

If an image is present then perform the action & return optional containing result;

otherwise return an empty optional


See docs.oracle.com/javase/8/docs/api/java/util/Optional.html#map

https://docs.oracle.com/javase/8/docs/api/java/util/Optional.html#map-java.util.function.Function-

47



return mFilters

.stream()



imageOpt

.map(image ->



getExecutor()));}

If an image is non-null then asynchronously filter the image & store it in an output file


48



return mFilters

.stream()



imageOpt

.map(image ->



getExecutor()));}

thenApplyAsync() runs the actions in a thread from the common fork-join pool


49



return mFilters

.stream()



imageOpt

.map(image ->



getExecutor()));

}

It also returns a new completable future that will trigger when the image has been filtered/stored


50




return mFilters

.stream()



imageOpt

.map(image ->



getExecutor()));

}

applyFiltersAsync() returns a stream of completable

futures to optional images

51






.stream()






log(stream.flatMap

(Optional::stream),

urls.size()))

.join();

flatMap() merges the stream of futures returned by applyFilters

Async() into a single stream

This stream is processed by collect(), as discussed in the next part of the lesson

52

End of Java 8 CompletableFutures ImageStreamGang

Example (Part 2)

java 8 completablefutures imagestreamgang example (part 2)schmidt/cs891f/2018-pdfs/...asynchronously...

Documents