simplifying concurrency: actors. in particular, akka

Simplifying concurrency: Actors In particular, AKKA

KRISTO IILA, ENGINEERING MANAGER

PLANET OS - MARCH 2016

The Many Ways of Concurrency

• Multithreading / semaphores / locking / mutexes

• Lock-free data structures

• Software Transactional Memory

• Message passing

• Futures / Async

• Actors


What are Actors?

• Lightweight object

• No shared state

• Communication via message passing

• Supervision hierarchies

• Originally implemented in Erlang (high availability telephone switches)

• Don’t need to think in terms of threads and locks

• Distributable by design


An Actor System Implementation - AKKA

• Implements actor model

• Makes remoting seamless

• Works with Java, Scala and .net

• Fits very well with scala

• Automatically scheduled to threads

• Can have multiple thread pools

Actor A Mailbox

Actor B

Message


Messages

• Communication between actors

• Tell: Fire and Forget

• Ask: Tell + Future { reply }

• Guaranteed ordering for one sender and receiver pair

• Access to a mailbox

• Messages can be anything

// Tell exampleactorRef ! Notify("hi!")

// Ask exampleval future = qaActor ? Question(“What?”)

// Process resultfuture.onComplete { … }

// Send result to actorfuture pipeTo actorRef2


Futures

• Future is a way to

Future { socket.read() }

retrieve results …

of concurrent operations …

asynchronously or synchronously.

• Examples

Eg code block:

Or actor ask: qaActor ? Question(“What is the meaning of life?”)

• Futures are composable


Actor Creation

• Each actor

val qaActor = system.actorOf( Props[QuestionAnswerer], “Oracle”)

... is an instance of a class with parameters

… has a name

• To create an actor:


Actor References

• You can not talk to actors directly

• When you create an actor, you get an ActorRef

• ActorRef is a reference to an actor… that can be passed around

… also in messages

… is serializable

… possibly remote

… hides local state


QA Actor example


But What if We Want an Already Existing ActorRef?

val selection = context.actorSelection("/user/AllKnowing/oracle")

• ActorSelection

val allAnswerers = context.actorSelection("/user/AllKnowing/*")

• The result can contain multiple actors:

allAnswerers ! Question(“Why are there petunias?”)

• But you can still send it messages:

• Transparently remote


But how do I know the actor path?

• Actors are in a hierarchy (tree)

• Tree root is ActorSystem

• System guardian and user guardian

• All user-created actors live under /user

• It is a bit more complicated if the actor is in a remote machine

ActorSystem

/user /system

/user/Allknowing

/user/AllKnowing/Oracle


What else can we do with the hierarchy? Supervision!

• Reliability

• Error kernel pattern

• Errors are escalated up the hierarchy

• Each level can decide how to respond

• Escalate error

• Restart child

• Stop child

• Resume child

• Top level guardian will shut down entire actor system

/c1

/b1

/c2

/a1

/b2

/user

/a2

/b3

/c4 /c5

/

/system


Where do messages go? Mailboxes!

• Each actor has access to one

• Possibly shared

• Multiple types

• Bounded / Unbounded

• Priority

• Can be order-preserving, but not always

Actor A Mailbox

Actor B

Message


Message Delivery Reliability

• Local message delivery is reliable Except for vm errors, and it’s better to always code for failure cases anyway.

• Mostly order preserving Unless you explicitly select non-order-preserving mailboxes of course

• Remote delivery is unreliable

• Should retry messages until acknowledged Or be ok with losing some, or most, or all, depending on the network.

• If message processing is idempotent, things get a lot easier to manage


Who schedules actors to threads? Dispatchers!

• Messages arrive to actor mailboxes

• Dispatcher takes a message

• from actor mailbox …

• and schedules actor …

• to handle it on a thread

• Threads are taken from execution contexts

• … aka thread pools

• Dispatchers are assigned to actors via rules

Actor A Mailbox

Actor B

Message

Dispatcher

Examples

Planet OS Data Intelligence


Websocket handling with actors

• Actor based web server

• One actor that receives connections

• Creates per-connection actors

• Supervises / terminates connection actors

• Connection actors handle data updates

• Better performance than thread-per-connection model

• Easier and scales better than … callbacks for data send/receive … event loops

ListeningSocket

Connection Manager Actor

ConnectionActor

ConnectionActor

ConnectionActor


Data update publish

Kafka

ConnectionActor

ConnectionActor

ConnectionActor

Kafkareceiver actor

Socket Connection

Socket Connection

Socket Connection


SCADA

SCADA

SCADA

WS Kafka Queue Process

Turbine 1

Turbine 2

Turbine 3

Power Plant

Dashboard


Event processing with actors

• Actors represent domain entities

• wind turbine

• power plant

• alert

• Receive event updates from Kafka

• Produce new events sent to Kafka

• Some events are joins of multiple events (eg alerts)

• Some events go to Web Sockets


Challenges

• Code must correctly handle idempotency

• Code must correctly handle crashes / restarts anywhere

• Data updates must not be lost

• Even cascaded data updates must not be lost

• Purely actor based model must mix consistency handling in business logic

Planet OS Data Hub


DPipe - the magic behind the Data Hub

• Data pipeline:

• Acquisition

• Transformation

• Analysis

• Indexing

• For spatio-temporal data:

• Time series

• Forecasts

• Satellites

• Sensors

• Models


DPipe - large scale data processing

• Multiple stages

• Each stage runs a cluster of actor systems

• Actor system has a scheduler that distributes work to worker actors

• Multiple pipelines also within a stage

• Can scale linearly with data sources

• Can do fork/join in data pipes with parallelisation

• Framework for exactly-once processing at scale

As if it wasn’t already quite obvious

We ARE in fact looking for new team members. There are many interesting challenges to work on.

Interested? Come talk to me or e-mail [email protected]

mailto:[email protected]

Questions?

KRISTO IILA, [email protected]

mailto:[email protected]