reactive revealed p1 - async nio, back-pressure and message- vs event-driven

Reactive Revealed: Message Drivenwith Konrad `@ktosopl` Malawski

Part #1

Message-driven Asynchronous

Non-blocking I/Oand Back-pressure

Part #2

ElasticityScalability

Location Transparency

Part #3

ResiliencyFailures vs Errors

Isolation & ContainmentDelegation & Replication

with Viktor Klang with Jonas Bonér


Konrad `ktoso` Malawski

Akka Team,Reactive Streams TCK

(we’re renaming soon!)


High Performance Software Development

For the majority of the time, high performance software development

is not about compiler hacks and bit twiddling.

It is about fundamental design principles that are key to doing any effective software development.

Martin Thompson

practicalperformanceanalyst.com/2015/02/17/getting-to-know-martin-thompson-...


http://www1.practicalperformanceanalyst.com/2015/02/17/getting-to-know-martin-thompson-high-performance-low-latency-computing-specialist/%23sthash.ce9OLwGx.dpuf

Agenda

• Reactive Manifesto & our focus today• Asynchronous vs. Synchronous• Messages compared to Events• Patterns enabled by Asynchronous

Messaging• Keeping things Non-blocking: Scheduling• Non-blocking I/O: IO, AIO, NIO, Zero• Asynchronous Non-blocking Back-

pressure• Wrapping up and Q/A


Initial version published 2 years ago.



A “not only Typesafe” initiative:Jonas Bonér, Dave Farley, Roland Kuhn, and Martin Thompson



A “not only Typesafe” initiative:Jonas Bonér, Dave Farley, Roland Kuhn, and Martin Thompson

Defines a tech agnostic shared vocabulary.



Asynchronous

Sync / Async Basics


Sync / Async Basics

Implicitly back-pressures the sender, since it can only send M2 after M1

finished.

(we’ll discuss back-pressure in depth shortly)


Sync / Async Basics



Messages compared to Events



Events are not addressed to a specific receiver.

They’re just “facts”, that get published to topics.



Event-based systems tend to focus on addressable event

sources.



A Message can contain an Event as it’s payload,

i.e. “listener ! SeatReserved()”.

Messages always have a specific recipient.

Be it Actor or Topic or something else.



Message-based systems tend to focus on addressable recipients.


Load management

Async enables transparent load balancing



Response latency grows since more and more messages pile up in the

mailbox…



Response latency grows since more and more messages pile up in the

mailbox…

We can:A. back-pressure the

PublisherB. add more workers!



Can add workers dynamically.


Non-blocking


Async where it matters: Scheduling

Scheduling (notice they grey sync call)

Scheduling (now with Async db call)

I / O


IO / AIO


IO / AIO / NIO


IO / AIO / NIO / Zero


Interruption!CPU: User Mode / Kernel Mode


Kernels and CPUs


Kernels and CPUs

http://wiki.osdev.org/Context_SwitchingReactive Revealed: Message Drivenwith Konrad `@ktosopl` Malawski

http://wiki.osdev.org/Context_Switching

Kernels and CPUs

[…] switching from user-level to kernel-level on a (2.8 GHz) P4 is 1348 cycles.

[…] Counting actual time, the P4 takes 481ns […]

http://wiki.osdev.org/Context_SwitchingReactive Revealed: Message Drivenwith Konrad `@ktosopl` Malawski


I / O


I / O

“Don’t worry. It only gets worse!”


I / O

“Don’t worry. It only gets worse!”

Same data in 3 buffers!4 mode switches!


Asynchronous I / O [Linux]

Linux AIO = JVM NIO



NewIO… since 2004!(No-one calls it “new” any more)

Linux AIO = JVM NIO



Less time wasted waiting.Same amount of buffer copies.


ZeroCopy = sendfile [Linux]

“Work smarter. Not harder.”

http://fourhourworkweek.com/Reactive Revealed: Message Drivenwith Konrad `@ktosopl` Malawski


Data never leaves kernel mode!


Delegating failures


Delegate Failures


Delegate Failures

Backup Requestsusing


Backup requests

A technique for fighting “long tail latencies”.By issuing duplicated work, when SLA seems in danger.


Backup requests


Backup requests - send


Backup requests

Avg Std dev 95%ile 99%ile 99.9%ile

No backups 33 ms 1524 ms 24 ms 52 ms 994 ms

After 10ms 14 ms 4 ms 20 ms 23 ms 50 ms

After 50ms 16 ms 12 ms 57 ms 63 ms 68 ms

Jeff Dean - Achieving Rapid Response Times in Large Online Services Peter Bailis - Doing Redundant Work to Speed Up Distributed Queries

Akka - Krzysztof Janosz @ Akkathon, Kraków - TailChoppingRouter (docs, pr)


http://static.googleusercontent.com/external_content/untrusted_dlcp/research.google.com/en/us/people/jeff/Berkeley-Latency-Mar2012.pdf

http://www.bailis.org/blog/doing-redundant-work-to-speed-up-distributed-queries/

http://doc.akka.io/docs/akka/snapshot/scala/routing.html%23TailChoppingPool_and_TailChoppingGroup

https://github.com/akka/akka/pull/15369

Lesson learned:

Backup requests allowtrade-off increased loadfor decreased latency.


Asynchronous Back-pressure


Back-pressure &

Combined Requests


Combined requests & back-pressure



No no no…!Not THAT Back-pressure!



THAT kind of back-pressure:

www.reactive-streams.orgReactive Revealed: Message Drivenwith Konrad `@ktosopl` Malawski

http://www.reactive-streams.org/


THAT kind of back-pressure:

Reactive Revealed: Message Drivenwith Konrad `@ktosopl` Malawskiwww.reactive-streams.org

http://www.reactive-streams.org/

Combined requests

A technique for avoiding duplicated work.By aggregating requests, possibly increasing latency.

“Wat? Why would I increase latency!?”


Combined requests


Lesson learned:

Back-pressure saves systems from overload.

Combined requests trade higher latency,for less work for the downstream.


Wrapping up


Wrapping up

• provides a shared vocabulary

• a “ubiquitous language“, for Reactive Systems

• is not re-discovering the wheel, instead just solidifies proven

patterns

• is technology independent

The Reactive Manifesto


Wrapping up

• are loosely coupled by design

• therefore can be elastic and resilient

• allow certain patterns to be used which otherwise wouldn’t be

possible

Asynchronous and Message-driven systems:


Links• akka.io• reactive-streams.org• akka-user

• Gil Tene - How NOT to measure latency, 2013• Jeff Dean @ Velocity 2014• Alan Bateman, Jeanfrancois Arcand (Sun) Async IO Tips @

JavaOne• http://linux.die.net/man/2/select• http://linux.die.net/man/2/poll• http://linux.die.net/man/4/epoll• giltene/jHiccup• Linux Journal: ZeroCopy I, Dragan Stancevis 2013

• Last slide car picture: http://actu-moteurs.com/sprint/gt-tour/jean-philippe-belloc-un-beau-challenge-avec-le-akka-asp-team/2000


http://akka.io/

http://reactive-streams.org/

https://groups.google.com/group/akka-user

http://www.slideshare.net/howarddgreen/how-not-to-measure-latency-london-oct-2013

https://www.youtube.com/watch?v=1-3Ahy7Fxsc

http://openjdk.java.net/projects/nio/presentations/TS-4222.pdf

http://openjdk.java.net/projects/nio/presentations/TS-4222.pdf

http://linux.die.net/man/2/select

http://linux.die.net/man/2/poll

http://linux.die.net/man/4/epoll

http://www.azulsystems.com/jhiccup

http://www.linuxjournal.com/article/6345?page=0,1

http://actu-moteurs.com/sprint/gt-tour/jean-philippe-belloc-un-beau-challenge-avec-le-akka-asp-team/2000

http://actu-moteurs.com/sprint/gt-tour/jean-philippe-belloc-un-beau-challenge-avec-le-akka-asp-team/2000

Links• http://wiki.osdev.org/Context_Switching• CppCon: Herb Sutter "Lock-Free Programming (or,

Juggling Razor Blades)"• http://www.infoq.com/presentations/reactive-services-scale• Gil Tene’s HdrHistogram.org

• http://hdrhistogram.github.io/HdrHistogram/plotFiles.html• Rob Pike - Concurrency is NOT Parallelism (video)• Brendan Gregg - Systems Performance: Enterprise and the Cloud (b

ook)• http://psy-lob-saw.blogspot.com/2015/02/hdrhistogram-better-laten

cy-capture.html• Jeff Dean, Luiz Andre Barroso - The Tail at Scale (whitepaper, ACM)• http://highscalability.com/blog/2012/3/12/google-taming-the-long-la

tency-tail-when-more-machines-equal.html• http://www.ulduzsoft.com/2014/01/select-poll-epoll-practical-differe

nce-for-system-architects/• Marcus Lagergren - Oracle JRockit: The Definitive Guide (book)• http://mechanical-sympathy.blogspot.com/2013/08/lock-based-vs-lo

ck-free-concurrent.html• Handling of Asynchronous Events -

http://www.win.tue.nl/~aeb/linux/lk/lk-12.html• http://www.kegel.com/c10k.html



https://www.youtube.com/watch?v=c1gO9aB9nbs



http://www.infoq.com/presentations/reactive-services-scale

http://hdrhistogram.org/

http://hdrhistogram.github.io/HdrHistogram/plotFiles.html

https://www.youtube.com/watch?v=cN_DpYBzKso

http://www.amazon.co.uk/gp/product/0133390098?psc=1&redirect=true&ref_=oh_aui_detailpage_o00_s00

http://www.amazon.co.uk/gp/product/0133390098?psc=1&redirect=true&ref_=oh_aui_detailpage_o00_s00

http://psy-lob-saw.blogspot.com/2015/02/hdrhistogram-better-latency-capture.html

http://psy-lob-saw.blogspot.com/2015/02/hdrhistogram-better-latency-capture.html

http://cacm.acm.org/magazines/2013/2/160173-the-tail-at-scale/abstract

http://highscalability.com/blog/2012/3/12/google-taming-the-long-latency-tail-when-more-machines-equal.html

http://highscalability.com/blog/2012/3/12/google-taming-the-long-latency-tail-when-more-machines-equal.html

http://www.ulduzsoft.com/2014/01/select-poll-epoll-practical-difference-for-system-architects/

http://www.ulduzsoft.com/2014/01/select-poll-epoll-practical-difference-for-system-architects/

http://www.amazon.com/Oracle-JRockit-Definitive-Marcus-Hirt-ebook/dp/B005CGIXLU/ref=asap_bc?ie=UTF8

http://mechanical-sympathy.blogspot.com/2013/08/lock-based-vs-lock-free-concurrent.html

http://mechanical-sympathy.blogspot.com/2013/08/lock-based-vs-lock-free-concurrent.html

http://www.win.tue.nl/~aeb/linux/lk/lk-12.html

http://www.kegel.com/c10k.html

Links• www.reactivemanifesto.org/• Seriously the only right way to micro benchmark on the

JVM:• JMH openjdk.java.net/projects/code-tools/jmh/• JMH for Scala: https://github.com/ktoso/sbt-jmh

• http://www.ibm.com/developerworks/library/l-async/• http://lse.sourceforge.net/io/aio.html• https://code.google.com/p/kernel/wiki/AIOUserGuide• ShmooCon: C10M - Defending the Internet At Scale (Robert

Graham)• http://blog.erratasec.com/2013/02/scalability-its-question-that-drives-us.html#.VO6E

11PF8SM

•User-level threads....... with threads. - Paul Turner @ Linux Plumbers Conf 2013

• https://www.facebook.com/themainstreetpiggyboys/photos/a.1390784047896753.1073741829.1390594984582326/1423592294615928/?type=1&theater for the Rollin’ Cuy on last slide


http://www.reactivemanifesto.org/

http://openjdk.java.net/projects/code-tools/jmh/

https://github.com/ktoso/sbt-jmh

http://www.ibm.com/developerworks/library/l-async/

http://lse.sourceforge.net/io/aio.html

https://code.google.com/p/kernel/wiki/AIOUserGuide

https://www.youtube.com/watch?v=D09jdbS6oSI

http://blog.erratasec.com/2013/02/scalability-its-question-that-drives-us.html%23.VO6E11PF8SM

http://blog.erratasec.com/2013/02/scalability-its-question-that-drives-us.html%23.VO6E11PF8SM

https://www.youtube.com/watch?v=KXuZi9aeGTw

https://www.youtube.com/watch?v=KXuZi9aeGTw

https://www.facebook.com/themainstreetpiggyboys/photos/a.1390784047896753.1073741829.1390594984582326/1423592294615928/?type=1&theater



Part #219 August 2015

ElasticityScalability

Location Transparency

Part #33 September 2015

ResiliencyFailures vs Errors

Isolation & ContainmentDelegation & Replication

with Viktor Klang with Jonas Bonér

Next time:


see blog.typesafe.com

http://blog.typesafe.com/

Go Reactive

More interesting stuff…

HOW TO VIDEO

Managing Library Dependencies with Play,

sbt & Activator

WATCH NOW

HAVE QUESTIONS?

Get in touch with Typesafe today!

CONTACT US

SUCCESS STORY

How Walmart Canada Saved Black Friday with

Akka, Play and Scala

READ NOW

http://www.typesafe.com/resources/case-studies-and-stories/walmart-boosts-conversions-by-20-with-typesafe-reactive-platform