Functional Reactive Programming with RxJava

Nguyen Cong Trung and Nguyen Hai Long Software Engineer at SeeSpace

Overview

1. Introduction to Reactive Programming.

2. Why should use Reactive Programming.

3. RxJava.

4. RxAndroid

5. Unit testing RxJava.

6. References.

7. Demo.

1. Introduction to Reactive Programming.

1.1. Side Effect.

1.2. Reactive Programming.

1.1. Side Effect.• In computer science, a function or expression is said to

have a side effect if, in addition to returning a value, it also modifies some state or has an observable interaction with calling functions or the outside world.

• For example, a particular function might modify a global variable or static variable, modify one of its arguments, raise an exception, write data to a display or file, read data, or call other side-effecting functions.

1.2. Reactive Programming.

• Reactive programming is programming with asynchronous data streams.

• Event buses or typical click events are really an asynchronous event stream, on which you can observe and do some side effects.

• Reactive is that idea on steroids. You are able to create data streams of anything, not just from click and hover events.

• It has many functions to combine, create and filter any of those streams.

• A stream can be used as an input to another one. Even multiple streams can be used as inputs to another stream.

• You can merge two streams. You can filter a stream to get another one that has only those events you are interested in. You can map data values from one stream to another new one.

• Each stream has many functions attached to it, such as map, filter, scan, etc. When you call one of these functions, such as clickStream.map(f), it returns a new stream based on the click stream. It does not modify the original click stream in any way. This is a property called immutability.

2. Why should use Reactive Programming.

2.1. Problems of Java Futures.

2.2. Problems of Callbacks.

2.3. Reactive.

2.1. Problems of Java Futures.

• Java Futures are straight-forward to use for a single level of asynchronous execution but they start to add non-trivial complexity when they're nested.

• Conditional asynchronous execution flows become difficult to optimally compose (particularly as latencies of each request vary at runtime) using Futures. It can be done of course, but it quickly becomes complicated (and thus error prone) or prematurely blocks on 'Future.get()', eliminating the benefit of asynchronous execution.

Link to demo.

2.2. Problems of Callbacks.

• Callbacks offer a solution to the tendency to block on Future.get() by not allowing anything to block. They are naturally efficient because they execute when the response is ready.

• Similar to Futures though, they are easy to use with a single level of asynchronous execution but become unwieldy with nested composition.

Link to demo.

2.3. Reactive.• Reactive programming offers efficient execution and

composition by providing a collection of operators capable of filtering, selecting, transforming, combining and composing Observable's.

• The Observable data type can be thought of as a "push" equivalent to Iterable which is "pull". With an Iterable, the consumer pulls values from the producer and the thread blocks until those values arrive. By contrast with the Observable type, the producer pushes values to the consumer whenever values are available. This approach is more flexible, because values can arrive synchronously or asynchronously.

• The following code demonstrates how a service layer method can choose whether to synchronously return data from an in-memory cache or asynchronously retrieve data from a remote service and callback with the data once retrieved. In both cases the client code consumes it the same way.

• The following code demonstrates the consumption of an Observable API.

• That code is declarative and lazy as well as functionally "pure" in that no mutation of state is occurring that would cause thread-safety issues.

• The API Service Layer is now free to change the behavior of the methods 'getListOfLists', 'getVideos', 'getMetadata', 'getBookmark' and 'getRating' – some blocking others non-blocking but all consumed the same way.

• In the example, 'getListOfLists' pushes each 'VideoList' object via 'onNext()' and then 'getVideos()' operates on that same parent thread. The implementation of that method could however change from blocking to non-blocking and the code would not need to change.

3. RxJava.

3.1. The Basics.

3.2. Hello, World!

3.3. Introducing Transformation and Operators.

3.4. More on Operators.

3.5. Error Handling.

3.6. Schedulers.

3.7. Subscriptions.

3.8. Backpressure.

3.9. Subject.

3.1. The Basics.• The basic building blocks of reactive code are Observables and

Subscribers. An Observable emits items; a Subscriber consumes those items.

• There is a pattern to how items are emitted. An Observable may emit any number of items (including zero items), then it terminates either by successfully completing, or due to an error.

• For each Subscriber it has, an Observable calls Subscriber.onNext() any number of times, followed by either Subscriber.onComplete() or Subscriber.onError().

• This looks a lot like your standard observer pattern, but it differs in one key way - Observables often don't start emitting items until someone explicitly subscribes to them.

3.2. Hello, World!

First, let's create a basic Observable:

Our Observable emits "Hello, world!" then completes. Now let's create a Subscriber to consume the data:

Now that we've got myObservable and mySubscriber we can hook them up to each other using subscribe():

RxJava has multiple built-in Observable creation methods for common tasks. In this case, Observable.just() emits a single item then completes, just like our code above:

Next, let's handle that unnecessarily verbose Subscriber. We don't care about onCompleted() nor onError(), so instead we can use a simpler class to define what to do during onNext():

Actions can define each part of a Subscriber. Observable.subscribe() can handle one, two or three Action parameters that take the place of onNext(), onError(), and onComplete(). Replicating our Subscriber from before looks like this:

However, we only need the first parameter, because we're ignoring onError() and onComplete():

Now, let's get rid of those variables by just chaining the method calls together:

Finally, let's use Java 8 lambdas to get rid of that ugly Action1 code.

3.3. Introducing Transformation and Operators.

Suppose I want to append my signature to the "Hello, world!" output. One possibility would be to change the Observable:

This works if you have control over your Observable, but there's no guarantee that will be the case - what if you're using someone else's library? Another potential problem: what if I use my Observable in multiple places but only sometimes want to add the signature?

3.3. Introducing Transformation and Operators.

How about we try modifying our Subscriber instead:

This answer is also unsatisfactory, but for different reasons: I want my Subscribers to be as lightweight as possible because I might be running them on the main thread. On a more conceptual level, Subscribers are supposed to be the thing that reacts, not the thing that mutates.

Wouldn't it be cool if I could transform "Hello, world!" with some intermediary step?

3.3. Introducing Transformation and Operators.

Operators can be used in between the source Observable and the ultimate Subscriber to manipulate emitted items. RxJava comes with a huge collection of operators, but its best to focus on just a handful at first.

For this situation, the map() operator can be used to transform one emitted item into another:

3.3. Introducing Transformation and Operators.

map() does not have to emit items of the same type as the source Observable.

Suppose my Subscriber is not interested in outputting the original text, but instead wants to output the hash of the text:

3.3. Introducing Transformation and Operators.

Again, we can use lambdas to shorten this code:

3.3. Introducing Transformation and Operators.

3.4. More on Operators.

Suppose I have this method available:

3.4. More on Operators.

I want to make a robust system for searching text and displaying the results. Given what we know from the last article, this is what one might come up with:

3.4. More on Operators.

This answer is highly unsatisfactory because I lose the ability to transform the data stream. If I wanted to modify each URL, I'd have to do it all in the Subscriber. We're tossing all our cool map() tricks out the window!

I could create a map() from urls -> urls, but then every map() call would have a for-each loop inside of it.

3.4. More on Operators.

There is a method, Observable.from(), that takes a collection of items and emits each them one at a time:

That looks like it could help, let's see what happens:

3.4. More on Operators.

I've gotten rid of the for-each loop, but the resulting code is a mess. I've got multiple, nested subscriptions now! Besides being ugly and hard to modify, it also breaks some critical as-yet undiscovered features of RxJava. The way RxJava does error handling, threading, and subscription cancellation wouldn't work at all with this code.

3.4. More on Operators.

Observable.flatMap() takes the emissions of one Observable and returns the emissions of another Observable to take its place. Here's how it solves this problem:

3.4. More on Operators.

Simplified with lambdas it looks awesome:

3.4. More on Operators.

flatMap() can return any Observable it wants. Suppose I've got a second method available:

Instead of printing the URLs, now I want to print the title of each website received. But there's a few issues: my method only works on a single URL at a time, and it doesn't return a String, it returns an Observable that emits the String.

3.4. More on Operators.

With flatMap(), solving this problem is easy; after splitting the list of URLs into individual items, I can use getTitle() in flatMap() for each url before it reaches the Subscriber:

3.4. More on Operators.

And once more, simplified via lambdas:

3.4. More on Operators.

We've only looked at two operators so far, but there are so many more! How else can we improve our code?

getTitle() returns null if the URL 404s. We don't want to output "null"; it turns out we can filter them out!

3.4. More on Operators.

filter() emits the same item it received, but only if it passes the boolean check.

And now we want to only show 5 results at most:

3.4. More on Operators.

take() emits, at most, the number of items specified. (If there are fewer than 5 titles it'll just stop early.)

Now we want to save each title to disk along the way:

3.4. More on Operators.

doOnNext() allows us to add extra behavior each time an item is emitted, in this case saving the title.

Link to other Operators.

3.5. Error Handling.Up until this point, we've largely been ignoring onComplete() and onError(). They mark when an Observable is going to stop emitting items and the reason for why (either a successful completion, or an unrecoverable error).

Our original Subscriber had the capability to listen to onComplete() and onError(). Let's actually do something with them:

3.5. Error Handling.

3.5. Error Handling.

• onError() is called if an Exception is thrown at any time.

• The operators don't have to handle the Exception.

• You know when the Subscriber has finished receiving items.

3.6. Schedulers.

• You've got an Android app that makes a network request. That could take a long time, so you load it in another thread. Suddenly, you've got problems!

• Multi-threaded Android applications are difficult because you have to make sure to run the right code on the right thread; mess up and your app can crash. The classic exception occurs when you try to modify a View off of the main thread.

3.6. Schedulers.• In RxJava, you can tell your Observer code which thread to run on

using subscribeOn(), and which thread your Subscriber should run on using observeOn():

3.6. Schedulers.• Everything that runs before my Subscriber runs on an I/O thread.

Then in the end, my View manipulation happens on the main thread.

• The great part about this is that I can attach subscribeOn() and observeOn() to any Observable! They're just operators! I don't have to worry about what the Observable or its previous operators are doing; I can just stick this at the end for easy threading.

3.6. Schedulers.You obtain a Scheduler from the factory methods described in the Schedulers class.

• Schedulers.computation( ): meant for computational work such as event-loops and callback processing; do not use this scheduler for I/O (use Schedulers.io( ) instead); the number of threads, by default, is equal to the number of processors.

• Schedulers.from(executor): uses the specified Executor as a Scheduler.

• Schedulers.immediate( ): schedules work to begin immediately in the current thread.

3.6. Schedulers.• Schedulers.io( ): meant for I/O-bound work such as asynchronous

performance of blocking I/O, this scheduler is backed by a thread-pool that will grow as needed; for ordinary computational work, switch to Schedulers.computation( ); Schedulers.io( ) by default is a CachedThreadScheduler, which is something like a new thread scheduler with thread caching.

• Schedulers.newThread( ): creates a new thread for each unit of work.

• Schedulers.trampoline( ): queues work to begin on the current thread after any already-queued work

3.7. Subscriptions.• When you call Observable.subscribe(), it returns a Subscription.

This represents the link between your Observable and your Subscriber:

3.7. Subscriptions.• You can use this Subscription to sever the link later on:

• What's nice about how RxJava handles unsubscribing is that it stops the chain. If you've got a complex chain of operators, using unsubscribe will terminate wherever it is currently executing code3. No unnecessary work needs to be done!

3.8. Backpressure.• In RxJava it is not difficult to get into a situation in which an

Observable is emitting items more rapidly than an operator or subscriber can consume them. This presents the problem of what to do with such a growing backlog of unconsumed items.

• The examples in this section will show how you might use such operators to handle a bursty Observable like the one illustrated in the following marble diagram:

• By fine-tuning the parameters to these operators you can ensure that a slow-consuming observer is not overwhelmed by a fast-producing Observable.

3.8. Backpressure.3.8.1. Throttling.

3.8.2. Buffers and windows.

3.8.1. Throttling.• Operators like sample( ) or throttleLast( ), throttleFirst( ), and

throttleWithTimeout( ) or debounce( ) allow you to regulate the rate at which an Observable emits items.

3.8.1. Throttling.Sample (or throttleLast):

The sample operator periodically "dips" into the sequence and emits only the most recently emitted item during each dip:

3.8.1. Throttling.throttleFirst

The throttleFirst operator is similar, but emits not the most recently emitted item, but the first item that was emitted after the previous "dip":

3.8.1. Throttling.debounce (or throttleWithTimeout)

The debounce operator emits only those items from the source Observable that are not followed by another item within a specified duration:

3.8.2. Buffers and windows.

You can also use an operator like buffer( ) or window( ) to collect items from the over-producing Observable and then emit them, less-frequently, as collections (or Observables) of items. The slow consumer can then decide whether to process only one particular item from each collection, to process some combination of those items, or to schedule work to be done on each item in the collection, as appropriate.

3.8.2. Buffers and windows.

buffer

You could, for example, close and emit a buffer of items from the bursty Observable periodically, at a regular interval of time:

3.8.2. Buffers and windows.

window

window is similar to buffer. One variant of window allows you to periodically emit Observable windows of items at a regular interval of time:

3.8.2. Buffers and windows.

3.8.2. Buffers and windows.

You could also choose to emit a new window each time you have collected a particular number of items from the source Observable:

3.8.2. Buffers and windows.

You could also choose to emit a new window each time you have collected a particular number of items from the source Observable:

3.9. Subject.3.9.1. The basics.

3.9.2. AsyncSubject.

3.9.3. BehaviorSubject.

3.9.4. PublishSubject.

3.9.5. ReplaySubject.

3.9.6. SerializedSubject.

3.9.1. The basics.A Subject is a sort of bridge or proxy that acts both as an Subscriber and as an Observable.

In this very basic example, I create a subject, subscribe to that subject and then publish values to the sequence (by calling subject.OnNext(T)).

If you have a Subject and you want to pass it along to some other agent without exposing its Subscriber interface, you can mask it by calling its asObservable method, which will return the Subject as a pure Observable.

3.9.1. The basics.

3.9.2. AsyncSubject.Subject that publishes only the last item observed to each Observer that has subscribed, when the source Observable completes.

3.9.3. BehaviorSubject.Subject that emits the most recent item it has observed and all subsequent observed items to each subscribed Observer.

3.9.3. BehaviorSubject.Example usage:

3.9.4. PublishSubject.Subject that, once an Observer has subscribed, emits all subsequently observed items to the subscriber.

3.9.4. PublishSubject.Example usage:

3.9.5. ReplaySubject.Subject that buffers all items it observes and replays them to any Observer that subscribes.

3.9.5. ReplaySubject.Example usage:

3.9.6. SerializedSubject.Wraps a Subject so that it is safe to call its various on methods from different threads.

When you use an ordinary Subject as a Subscriber, you must take care not to call its Observer.onNext(T) method (or its other on methods) from multiple threads, as this could lead to non-serialized calls, which violates the Observable contract and creates an ambiguity in the resulting Subject.

To protect a Subject from this danger, you can convert it into a SerializedSubject with code like the following:

4. RxAndroid.

4.1. Overview.

4.2. Retrofit.

4.3. Migrate old code to Observable.

4.4. Lifecycle.

4.1. Overview.RxAndroid is an extension to RxJava built just for Android. It includes special bindings that will make your life easier.

First, there's AndroidSchedulers which provides schedulers ready-made for Android's threading system. Need to run some code on the UI thread? No problem - just use AndroidSchedulers.mainThread():

4.1. Overview.If you've got your own Handler, you can create a scheduler linked to it with HandlerThreadScheduler.

Next we have AndroidObservable which provides more facilities for working within the Android lifecycle. There is bindActivity() and bindFragment() which, in addition to automatically using AndroidSchedulers.mainThread() for observing, will also stop emitting items when your Activity or Fragment is finishing (so you don't accidentally try to change state after it is valid to do so).

4.1. Overview.

I also like AndroidObservable.fromBroadcast(), which allows you to create an Observable that works like a BroadcastReceiver. Here's a way to be notified whenever network connectivity changes:

4.1. Overview.Finally, there is ViewObservable, which adds a couple bindings for Views. There's ViewObservable.clicks() if you want to get an event each time a View is clicked, or ViewObservable.text() to observe whenever a TextView changes its content.

4.2. Retrofit.There's one notable library that supports RxJava: Retrofit, a popular REST client for Android. Normally when you define an asynchronous method you add a Callback:

With RxJava installed, you can have it return an Observable instead:

4.2. Retrofit.Retrofit support for Observable also makes it easy to combine multiple REST calls together. For example, suppose we have one call that gets the photo and a second that gets the metadata. We can zip the results together:

4.3. Migrate old code to Observable.

Observable.just() and Observable.from() are suffice for creating an Observable from older code most of the time:

That works well if oldMethod() is fast, but what if it's slow? It'll block the thread because you're calling oldMethod() before passing it to Observable.just().

4.3. Migrate old code to Observable.

To get around that problem, here's a trick I use all the time - wrapping the slower part with defer():

Now, the Observable returned won't call slowBlockingMethod() until you subscribe to it.

4.4. Lifecycle.How do you handle the Activity lifecycle? There are two issues that crop up over and over again:

1. Continuing a Subscription during a configuration change (e.g. rotation). Suppose you make REST call with Retrofit and then want to display the outcome in a ListView. What if the user rotates the screen? You want to continue the same request, but how?

2. Memory leaks caused by Observables which retain a copy of the Context. This problem is caused by creating a subscription that retains the Context somehow, which is not difficult when you're interacting with Views! If Observable doesn't complete on time, you may end up retaining a lot of extra memory.

4.4. Lifecycle.The first problem can be solved with some of RxJava's built-in caching mechanisms, so that you can unsubscribe/resubscribe to the same Observable without it duplicating its work. In particular, cache() (or replay()) will continue the underlying request (even if you unsubscribe). That means you can resume with a new subscription after Activity recreation:

4.4. Lifecycle.Note that we're using the same cached request in both cases; that way the underlying call only happens once. Where you store request I leave up to you, but like all lifecycle solutions, it must be stored somewhere outside the lifecycle (a retained fragment, a singleton, etc).

4.4. Lifecycle.The second problem can be solved by properly unsubscribing from your subscriptions in accordance with the lifecycle. It's a common pattern to use a CompositeSubscription to hold all of your Subscriptions, and then unsubscribe all at once in onDestroy() or onDestroyView():

4.4. Lifecycle.

4.4. Lifecycle.For bonus points you can create a root Activity/Fragment that comes with a CompositeSubscription that you can add to and is later automatically unsubscribed.

A warning! Once you call CompositeSubscription.unsubscribe() the object is unusable, as it will automatically unsubscribe anything you add to it afterwards! You must create a new CompositeSubscription as a replacement if you plan on re-using this pattern later.

5. Unit testing RxJava.The ugly way

The first way is to simply subscribe in the same way we would from outside the tests and then save the result in a global variable that can later on be asserted.

For example, imagine we have a method in a database helper class that is in charge of loading a user object.

5. Unit testing RxJava.Then the test for this method would look like this:

5. Unit testing RxJava.This code works because, by default, the Observable will run on the same thread. Therefore, the assertion will always happen after the result is set in the global variable.

5. Unit testing RxJava.The better way

Use static methods of a class called RxAssertions written by Software Engineers at ribot.

5. Unit testing RxJava.The official way

Use TestSubscriber provided by RxJava.

Chaining assertions is not possible, because the assert methods don’t return the TestSubscriber.

6. References.• Side effect (computer science) – Wikipedia.

• Referential transparency (computer science) – Wikipedia.

• The introduction to Reactive Programming you've been missing by André Staltz.

• Reactive Programming in the Netflix API with RxJava by Ben Christensen and Jafar Husain.

• Grokking RxJava by Dan Lew.

• Unit Testing RxJava Observables by Iván Carballo.

• Learning RxJava for Android by example by Kaushik Gopal.

• ReactiveX IO.

• Official RxJava’s Wiki.

7. Demo.