functional reactive programming in the netflix api · 2014. 9. 25. · functional reactive...
TRANSCRIPT
![Page 1: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/1.jpg)
Functional Reactive Programmingin the Netflix API
Ben ChristensenSoftware Engineer – API Platform at Netflix@benjchristensenhttp://www.linkedin.com/in/benjchristensen
http://techblog.netflix.com/
QCon London – March 6 2013
onsdag den 6. marts 13
![Page 2: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/2.jpg)
onsdag den 6. marts 13
![Page 3: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/3.jpg)
More than 33 million Subscribersin more than 50 Countries and Territories
onsdag den 6. marts 13
![Page 4: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/4.jpg)
Netflix accounts for 33% of Peak Downstream Internet Traffic in North America
Netflix subscribers are watching more than 1 billion hours a month
onsdag den 6. marts 13
![Page 5: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/5.jpg)
API traffic has grown from ~20 million/day in 2010 to >2 billion/day
0
500
1000
1500
2000
2010 2011 2012 Today
milli
ons
of A
PI re
ques
ts p
er d
ay
onsdag den 6. marts 13
![Page 6: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/6.jpg)
Discovery Streaming
onsdag den 6. marts 13
![Page 7: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/7.jpg)
Netflix API Streaming
onsdag den 6. marts 13
![Page 8: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/8.jpg)
onsdag den 6. marts 13
![Page 9: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/9.jpg)
Open API Netflix Devices
API Request Volume by Audience
onsdag den 6. marts 13
![Page 10: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/10.jpg)
Netflix API
Dependency A
Dependency D
Dependency G
Dependency J
Dependency M
Dependency P
Dependency B
Dependency E
Dependency H
Dependency K
Dependency N
Dependency Q
Dependency C
Dependency F
Dependency I
Dependency L
Dependency O
Dependency R
onsdag den 6. marts 13
![Page 11: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/11.jpg)
/ps3
/hom
e
Dependency F10 Threads
Dependency G10 Threads
Dependency H10 Threads
Dependency I5 Threads
Dependency J8 Threads
Dependency A10 Threads
Dependency B8 Threads
Dependency C10 Threads
Dependency D15 Threads
Dependency E5 Threads
Dependency K15 Threads
Dependency L4 Threads
Dependency M5 Threads
Dependency N10 Threads
Dependency O10 Threads
Dependency P10 Threads
Dependency Q8 Threads
Dependency R10 Threads
Dependency S 8 Threads
Dependency T10 Threads
/and
roid
/hom
e
/tv/h
ome
Functional Reactive Dynamic Endpoints
Asynchronous Java API
onsdag den 6. marts 13
![Page 12: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/12.jpg)
/ps3
/hom
e
Dependency F10 Threads
Dependency G10 Threads
Dependency H10 Threads
Dependency I5 Threads
Dependency J8 Threads
Dependency A10 Threads
Dependency B8 Threads
Dependency C10 Threads
Dependency D15 Threads
Dependency E5 Threads
Dependency K15 Threads
Dependency L4 Threads
Dependency M5 Threads
Dependency N10 Threads
Dependency O10 Threads
Dependency P10 Threads
Dependency Q8 Threads
Dependency R10 Threads
Dependency S 8 Threads
Dependency T10 Threads
/and
roid
/hom
e
/tv/h
ome
Functional Reactive Dynamic Endpoints
Asynchronous Java API
Hystrix fault-isolation layer
onsdag den 6. marts 13
![Page 13: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/13.jpg)
/ps3
/hom
e
Dependency F10 Threads
Dependency G10 Threads
Dependency H10 Threads
Dependency I5 Threads
Dependency J8 Threads
Dependency A10 Threads
Dependency B8 Threads
Dependency C10 Threads
Dependency D15 Threads
Dependency E5 Threads
Dependency K15 Threads
Dependency L4 Threads
Dependency M5 Threads
Dependency N10 Threads
Dependency O10 Threads
Dependency P10 Threads
Dependency Q8 Threads
Dependency R10 Threads
Dependency S 8 Threads
Dependency T10 Threads
/and
roid
/hom
e
/tv/h
ome
Functional Reactive Dynamic Endpoints
Asynchronous Java API
onsdag den 6. marts 13
![Page 14: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/14.jpg)
RxJava
“a library for composing asynchronous and event-based
programs using observable sequences for the Java VM”
A Java port of Rx (Reactive Extensions)https://rx.codeplex.com (.Net and Javascript by Microsoft)
onsdag den 6. marts 13
![Page 15: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/15.jpg)
Do we really need another way of“managing” concurrency?
onsdag den 6. marts 13
![Page 16: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/16.jpg)
Discovery of Rx began with a re-architecture ...
onsdag den 6. marts 13
![Page 17: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/17.jpg)
... that collapsed network traffic into coarse API calls ...
onsdag den 6. marts 13
![Page 18: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/18.jpg)
... that collapsed network traffic into coarse API calls ...
Nested, conditional, parallel execution
onsdag den 6. marts 13
![Page 19: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/19.jpg)
... and we wanted to allow
anybody to create endpoints,
not just the “API Team”
onsdag den 6. marts 13
![Page 20: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/20.jpg)
onsdag den 6. marts 13
![Page 21: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/21.jpg)
Concurrency without each engineer reading and re-reading this ->
(awesome book ... everybody isn’t going to - or should have to - read
it though, that’s the point)
onsdag den 6. marts 13
![Page 22: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/22.jpg)
Owner of API should retain control of concurrency behavior.
onsdag den 6. marts 13
![Page 23: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/23.jpg)
public Data getData();
What if the implementation needs to change from synchronous to asynchronous?
How should the client execute that method without blocking? spawn a thread?
Owner of API should retain control of concurrency behavior.
onsdag den 6. marts 13
![Page 24: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/24.jpg)
public void getData(Callback<T> c);
public Future<T> getData();
public Future<List<Future<T>>> getData();
What about ... ?
onsdag den 6. marts 13
![Page 25: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/25.jpg)
Iterablepull
Observablepush
T next()throws Exception
returns;
onNext(T)onError(Exception)onCompleted()
onsdag den 6. marts 13
![Page 26: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/26.jpg)
Iterablepull
Observablepush
T next()throws Exception
returns;
onNext(T)onError(Exception)onCompleted()
// Iterable<String> // that contains 75 Strings getDataFromLocalMemory() .skip(10) .take(5) .map({ s -‐> return s + "_transformed"}) .forEach( { println "next => " + it})
// Observable<String> // that emits 75 Strings getDataFromNetwork() .skip(10) .take(5) .map({ s -‐> return s + "_transformed"}) .subscribe( { println "onNext => " + it})
onsdag den 6. marts 13
![Page 27: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/27.jpg)
Iterablepull
Observablepush
T next()throws Exception
returns;
onNext(T)onError(Exception)onCompleted()
// Iterable<String> // that contains 75 Strings getDataFromLocalMemory() .skip(10) .take(5) .map({ s -‐> return s + "_transformed"}) .forEach( { println "onNext => " + it})
// Observable<String> // that emits 75 Strings getDataFromNetwork() .skip(10) .take(5) .map({ s -‐> return s + "_transformed"}) .subscribe( { println "onNext => " + it})
onsdag den 6. marts 13
![Page 28: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/28.jpg)
Instead of blocking APIs ...class VideoService { def VideoList getPersonalizedListOfMovies(userId); def VideoBookmark getBookmark(userId, videoId); def VideoRating getRating(userId, videoId); def VideoMetadata getMetadata(videoId);}
class VideoService { def Observable<VideoList> getPersonalizedListOfMovies(userId); def Observable<VideoBookmark> getBookmark(userId, videoId); def Observable<VideoRating> getRating(userId, videoId); def Observable<VideoMetadata> getMetadata(videoId);}
... create Observable APIs:
onsdag den 6. marts 13
![Page 29: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/29.jpg)
onsdag den 6. marts 13
![Page 30: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/30.jpg)
onsdag den 6. marts 13
![Page 31: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/31.jpg)
onsdag den 6. marts 13
![Page 32: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/32.jpg)
onsdag den 6. marts 13
![Page 33: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/33.jpg)
onsdag den 6. marts 13
![Page 34: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/34.jpg)
onsdag den 6. marts 13
![Page 35: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/35.jpg)
Observable.toObservable("one", "two", "three") .take(2) .subscribe((arg) -‐> { System.out.println(arg); });
Java8
Observable.toObservable("one", "two", "three") .take(2) .subscribe((arg: String) => { println(arg) })
Scala(-‐> (Observable/toObservable ["one" "two" "three"]) (.take 2) (.subscribe (fn [arg] (println arg))))
Clojure
Observable.toObservable("one", "two", "three") .take(2) .subscribe({arg -‐> println(arg)})
Groovy Observable.toObservable("one", "two", "three") .take(2) .subscribe(lambda { |arg| puts arg })
JRuby
onsdag den 6. marts 13
![Page 36: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/36.jpg)
Observable.create({ observer -‐> try { observer.onNext(new Video(id)) observer.onCompleted(); } catch(Exception e) { observer.onError(e); } })
onsdag den 6. marts 13
![Page 37: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/37.jpg)
def Observable<VideoRating> getRating(userId, videoId) { // fetch the VideoRating for this user asynchronously return Observable.create({ observer -‐> executor.execute(new Runnable() { def void run() { try { VideoRating rating = ... do network call ... observer.onNext(rating) observer.onCompleted(); } catch(Exception e) { observer.onError(e); } } }) }) }
Asynchronous Observable with Single Value
onsdag den 6. marts 13
![Page 38: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/38.jpg)
def Observable<VideoRating> getRating(userId, videoId) { // fetch the VideoRating for this user asynchronously return Observable.create({ observer -‐> executor.execute(new Runnable() { def void run() { try { VideoRating rating = ... do network call ... observer.onNext(rating) observer.onCompleted(); } catch(Exception e) { observer.onError(e); } } }) }) }
Asynchronous Observable with Single Value
onsdag den 6. marts 13
![Page 39: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/39.jpg)
Synchronous Observable with Multiple Values
def Observable<Video> getVideos() { return Observable.create({ observer -‐> try { for(v in videos) { observer.onNext(v) } observer.onCompleted(); } catch(Exception e) { observer.onError(e); } }) }
Caution: This is eager and will always emit all values regardless of subsequent operators such as take(10)
onsdag den 6. marts 13
![Page 40: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/40.jpg)
Synchronous Observable with Multiple Values
def Observable<Video> getVideos() { return Observable.create({ observer -‐> try { for(v in videos) { observer.onNext(v) } observer.onCompleted(); } catch(Exception e) { observer.onError(e); } }) }
Caution: This is eager and will always emit all values regardless of subsequent operators such as take(10)
onsdag den 6. marts 13
![Page 41: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/41.jpg)
Asynchronous Observable with Multiple Values def Observable<Video> getVideos() { return Observable.create({ observer -‐> executor.execute(new Runnable() { def void run() { try { for(id in videoIds) { Video v = ... do network call ... observer.onNext(v) } observer.onCompleted(); } catch(Exception e) { observer.onError(e); } } }) }) }
onsdag den 6. marts 13
![Page 42: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/42.jpg)
Asynchronous Observable with Multiple Values def Observable<Video> getVideos() { return Observable.create({ observer -‐> executor.execute(new Runnable() { def void run() { try { for(id in videoIds) { Video v = ... do network call ... observer.onNext(v) } observer.onCompleted(); } catch(Exception e) { observer.onError(e); } } }) }) }
onsdag den 6. marts 13
![Page 43: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/43.jpg)
Observable<SomeData> a = getDataA();Observable<SomeData> b = getDataB();Observable<SomeData> c = getDataC();
Observable.merge(a, b, c) .subscribe( { element -> println("data: " + element)}, { exception -> println("error occurred: " + exception.getMessage())} )
Combining via Merge
onsdag den 6. marts 13
![Page 44: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/44.jpg)
Observable<SomeData> a = getDataA();Observable<String> b = getDataB();Observable<MoreData> c = getDataC();
Observable.zip(a, b, c, {x, y, z -> [x, y, z]}) .subscribe( { triple -> println("a: " + triple[0] + " b: " + triple[1] + " c: " + triple[2])}, { exception -> println("error occurred: " + exception.getMessage())} )
Combining via Zip
onsdag den 6. marts 13
![Page 45: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/45.jpg)
Observable<SomeData> a = getDataA();Observable<String> b = getDataB();Observable<MoreData> c = getDataC();
Observable.zip(a, b, c, {x, y, z -> [x, y, z]}) .subscribe( { triple -> println("a: " + triple[0] + " b: " + triple[1] + " c: " + triple[2])}, { exception -> println("error occurred: " + exception.getMessage())} )
Error Handling
onsdag den 6. marts 13
![Page 46: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/46.jpg)
Observable<SomeData> a = getDataA();Observable<String> b = getDataB();Observable<MoreData> c = getDataC() .onErrorResumeNext(getFallbackForDataC());
Observable.zip(a, b, c, {x, y, z -> [x, y, z]}) .subscribe( { triple -> println("a: " + triple[0] + " b: " + triple[1] + " c: " + triple[2])}, { exception -> println("error occurred: " + exception.getMessage())} )
Error Handling
onsdag den 6. marts 13
![Page 47: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/47.jpg)
def Observable getVideos(userId) { return VideoService.getVideos(userId)
}
onsdag den 6. marts 13
![Page 48: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/48.jpg)
def Observable getVideos(userId) { return VideoService.getVideos(userId)
}
Asynchronous request that returns Observable<Video>
onsdag den 6. marts 13
![Page 49: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/49.jpg)
def Observable<Map> getVideos(userId) { return VideoService.getVideos(userId) // we only want the first 10 of each list .take(10)}
onsdag den 6. marts 13
![Page 50: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/50.jpg)
def Observable<Map> getVideos(userId) { return VideoService.getVideos(userId) // we only want the first 10 of each list .take(10)}
Reactive operator on the Observablethat takes the first 10 Video objects
then unsubscribes.
onsdag den 6. marts 13
![Page 51: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/51.jpg)
def Observable<Map> getVideos(userId) { return VideoService.getVideos(userId) // we only want the first 10 of each list .take(10) .map({ Video video -> // transform video object }) }
onsdag den 6. marts 13
![Page 52: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/52.jpg)
def Observable<Map> getVideos(userId) { return VideoService.getVideos(userId) // we only want the first 10 of each list .take(10) .map({ Video video -> // transform video object }) }
The ‘map’ operator allows transformingthe input value into a different output.
onsdag den 6. marts 13
![Page 53: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/53.jpg)
Observable<R> b = Observable<T>.map({ T t -‐> R r = ... transform t ... return r; })
onsdag den 6. marts 13
![Page 54: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/54.jpg)
def Observable<Map> getVideos(userId) { return VideoService.getVideos(userId) // we only want the first 10 of each list .take(10) .mapMany({ Video video -> // for each video we want to fetch metadata def m = video.getMetadata() .map({ Map<String, String> md -> // transform to the data and format we want return [title: md.get("title"), length: md.get("duration")] }) // and its rating and bookmark def b ... def r ... }) }
onsdag den 6. marts 13
![Page 55: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/55.jpg)
def Observable<Map> getVideos(userId) { return VideoService.getVideos(userId) // we only want the first 10 of each list .take(10) .mapMany({ Video video -> // for each video we want to fetch metadata def m = video.getMetadata() .map({ Map<String, String> md -> // transform to the data and format we want return [title: md.get("title"), length: md.get("duration")] }) // and its rating and bookmark def b ... def r ... }) } We change to ‘mapMany’ which is
like merge(map()) since we will return an Observable<T> instead of T.
onsdag den 6. marts 13
![Page 56: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/56.jpg)
Observable<R> b = Observable<T>.mapMany({ T t -‐> Observable<R> r = ... transform t ... return r; })
onsdag den 6. marts 13
![Page 57: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/57.jpg)
Observable<R> b = Observable<T>.mapMany({ T t -‐> Observable<R> r = ... transform t ... return r; })
onsdag den 6. marts 13
![Page 58: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/58.jpg)
def Observable<Map> getVideos(userId) { return VideoService.getVideos(userId) // we only want the first 10 of each list .take(10) .mapMany({ Video video -> // for each video we want to fetch metadata def m = video.getMetadata() .map({ Map<String, String> md -> // transform to the data and format we want return [title: md.get("title"), length: md.get("duration")] }) // and its rating and bookmark def b ... def r ... }) }
Nested asynchronous callsthat return more Observables.
onsdag den 6. marts 13
![Page 59: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/59.jpg)
def Observable<Map> getVideos(userId) { return VideoService.getVideos(userId) // we only want the first 10 of each list .take(10) .mapMany({ Video video -> // for each video we want to fetch metadata def m = video.getMetadata() .map({ Map<String, String> md -> // transform to the data and format we want return [title: md.get("title"), length: md.get("duration")] }) // and its rating and bookmark def b ... def r ... }) }
Observable<VideoMetadata>Observable<VideoBookmark>Observable<VideoRating>
onsdag den 6. marts 13
![Page 60: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/60.jpg)
def Observable<Map> getVideos(userId) { return VideoService.getVideos(userId) // we only want the first 10 of each list .take(10) .mapMany({ Video video -> // for each video we want to fetch metadata def m = video.getMetadata() .map({ Map<String, String> md -> // transform to the data and format we want return [title: md.get("title"), length: md.get("duration")] }) // and its rating and bookmark def b ... def r ... }) }
Each Observable transformsits data using ‘map’
onsdag den 6. marts 13
![Page 61: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/61.jpg)
def Observable<Map> getVideos(userId) { return VideoService.getVideos(userId) // we only want the first 10 of each list .take(10) .mapMany({ Video video -> // for each video we want to fetch metadata def m = video.getMetadata() .map({ Map<String, String> md -> // transform to the data and format we want return [title: md.get("title"), length: md.get("duration")] }) // and its rating and bookmark def b ... def r ... // compose these together }) }
onsdag den 6. marts 13
![Page 62: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/62.jpg)
def Observable<Map> getVideos(userId) { return VideoService.getVideos(userId) // we only want the first 10 of each list .take(10) .mapMany({ Video video -> def m ... def b ... def r ... // compose these together }) }
onsdag den 6. marts 13
![Page 63: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/63.jpg)
def Observable<Map> getVideos(userId) { return VideoService.getVideos(userId) // we only want the first 10 of each list .take(10) .mapMany({ Video video -> def m ... def b ... def r ... // compose these together return Observable.zip(m, b, r, { metadata, bookmark, rating -> // now transform to complete dictionary // of data we want for each Video return [id: video.videoId] << metadata << bookmark << rating }) }) }
onsdag den 6. marts 13
![Page 64: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/64.jpg)
def Observable<Map> getVideos(userId) { return VideoService.getVideos(userId) // we only want the first 10 of each list .take(10) .mapMany({ Video video -> def m ... def b ... def r ... // compose these together return Observable.zip(m, b, r, { metadata, bookmark, rating -> // now transform to complete dictionary // of data we want for each Video return [id: video.videoId] << metadata << bookmark << rating }) }) }
The ‘zip’ operator combines the 3 asynchronous Observables into 1
onsdag den 6. marts 13
![Page 65: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/65.jpg)
Observable.zip(a, b, { a, b, -‐> ... operate on values from both a & b ... return [a, b]; // i.e. return tuple })
onsdag den 6. marts 13
![Page 66: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/66.jpg)
def Observable<Map> getVideos(userId) { return VideoService.getVideos(userId) // we only want the first 10 of each list .take(10) .mapMany({ Video video -> def m ... def b ... def r ... // compose these together return Observable.zip(m, b, r, { metadata, bookmark, rating -> // now transform to complete dictionary // of data we want for each Video return [id: video.videoId] << metadata << bookmark << rating }) }) }
return a single Map (dictionary) of transformed and combined data
from 4 asynchronous callsonsdag den 6. marts 13
![Page 67: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/67.jpg)
onsdag den 6. marts 13
![Page 68: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/68.jpg)
Observable<Video> emits n videos to onNext()onsdag den 6. marts 13
![Page 69: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/69.jpg)
Takes first 10 then unsubscribes from origin.Returns Observable<Video> that emits 10 Videos.
onsdag den 6. marts 13
![Page 70: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/70.jpg)
For each of the 10 Video objects it transformsvia ‘mapMany’ function that does nested async calls.
onsdag den 6. marts 13
![Page 71: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/71.jpg)
For each Video ‘v’ it calls getMetadata()which returns Observable<VideoMetadata>
These nested async requests return Observables
that emit 1 value.
onsdag den 6. marts 13
![Page 72: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/72.jpg)
The Observable<VideoMetadata> is transformed via a ‘map’ function to return a Map of key/values.
onsdag den 6. marts 13
![Page 73: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/73.jpg)
Same for Observable<VideoBookmark> and Observable<VideoRating>
onsdag den 6. marts 13
![Page 74: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/74.jpg)
The 3 ‘mapped’ Observables are combined with a ‘zip’ function that emits a Map with all data.
onsdag den 6. marts 13
![Page 75: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/75.jpg)
The full sequence emits Observable<Map> that emits a Map for each of 10 Videos.
onsdag den 6. marts 13
![Page 76: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/76.jpg)
Client code treats all interactions with the API as asynchronous
The API implementation chooses whether something is
blocking or non-blocking and
what resources it uses.
onsdag den 6. marts 13
![Page 77: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/77.jpg)
Example of latency reduction achieved by increasing number of threads used by Observables.
onsdag den 6. marts 13
![Page 78: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/78.jpg)
+
Observable<User> u = new GetUserCommand(id).observe();Observable<Geo> g = new GetGeoCommand(request).observe();
Observable.zip(u, g, {user, geo -‐> return [username: user.getUsername(), currentLocation: geo.getCounty()] })
RxJava coming to Hystrixhttps://github.com/Netflix/Hystrix
onsdag den 6. marts 13
![Page 79: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/79.jpg)
<dependency org="com.netflix.rxjava" name="rxjava-‐core" rev="x.y.z" />
<dependency org="com.netflix.rxjava" name="rxjava-‐groovy" rev="x.y.z" /><dependency org="com.netflix.rxjava" name="rxjava-‐clojure" rev="x.y.z" /><dependency org="com.netflix.rxjava" name="rxjava-‐scala" rev="x.y.z" /><dependency org="com.netflix.rxjava" name="rxjava-‐jruby" rev="x.y.z" />
<dependency> <groupId>com.netflix.rxjava</groupId> <artifactId>rxjava-‐core</artifactId> <version>x.y.z</version></dependency>
... or for a different language ...
To get started ...
onsdag den 6. marts 13
![Page 80: Functional Reactive Programming in the Netflix API · 2014. 9. 25. · Functional Reactive Programming in the Netflix API Ben Christensen Software Engineer – API Platform at Netflix](https://reader033.vdocument.in/reader033/viewer/2022053119/60a161d0451532519f57532b/html5/thumbnails/80.jpg)
Functional Reactive in the Netflix API with RxJavahttp://techblog.netflix.com/2013/02/rxjava-netflix-api.html
Optimizing the Netflix APIhttp://techblog.netflix.com/2013/01/optimizing-netflix-api.html
RxJavahttps://github.com/Netflix/RxJava
@RxJava
Ben Christensen@benjchristensen
http://www.linkedin.com/in/benjchristensen
Netflix is Hiringhttp://jobs.netflix.com
onsdag den 6. marts 13