Real World Scala hAkking

Raymond Roestenburg

About me

• Java since 1997• Scala since 2010• Mostly backend systems• Lead architect Traffic Management at CSC• Akka committer

code: http://github.com/RayRoestenburg

blog: http://roestenburg.agilesquad.com

twtr: @RayRoestenburg

Company I work for

Traffic Management• Section Control• Weigh in Motion• Traffic Information• Traffic Enforcement

Agenda

• @MIGO-BORAS sensordomain• Sensor networks• Typesafe stack• Akka Actors• Apache Camel Integration• Remote Actors• Monitoring• Testing• Deployment

@MIGO-BORAS

• Koninklijke Marechaussee• MTV (Mobiel Toezicht Vreemdelingen)• Prevent illegal border crossing• Sensor domain• Fixed locations• Mobile vehicles• Cameras, Radars• Vehicle properties• January 2012

Sensor networks

• Usable for many applications in our field• Distributed along highways and provincial roads• Traffic Enforcement• Traffic Information• Other applications• Roadside systems• Range of sensor types

– Camera’s, Infrared, Laser, Piezoelectric, Loops, Mobile units, …

• Remotely Configure & Control

Why Scala & Akka

• Less code, less bugs• Less code, same VM power• Experienced team (Java, C#, C)

• 3 experienced Java devs • 1 experienced C# dev with no Java experience

• Need to interoperate with existing Java and C (JNA)

• Simplify development (C->Java->Scala)• Ubuntu

Scala 2.9.x

• Learning curve (Start with simple features)• Traits, Tupels, Stackable Traits• First Class Functions• Scala Collections

• map, flatMap, partition, foreach, filter, groupBy..

• Option type• map.. orElse, foreach,..

• Case classes • REPL, DSLs, Extension methods

Typesafe Stack

• Current Stack• Open Source• SBT 0.7.7• Scala 2.9.0-1• Akka 1.1.3• Camel & Camel Components 2.7.0

(FTP,Mina,Jetty,…)• Jboss Netty 3.2.4

• When we started early 2010• Akka 0.7, Scala 2.8.0Beta1

Akka Usage

Core

Actors, Remote, TestKit, Serialization, STM, Agents, Dataflow, Async HTTP (Mist), FSM, Transactors

Modules

MicroKernel, Camel, AMQP, Scalaz, Spring, OSGi, DataFlow, Persistence,

Akka Actors

Dispatcher

ActorRef Mailbox Actor

Threads

val a = actorOf(new Act())

a ! msgMessage to send

Akka Actors

Dispatcher

Mailbox

Threads

ActorActorRef

class Act extends Actor { def receive = { case msg:Msg => { // we are here } }}

Akka Actors

Dispatcher

Mailbox

Threads

ActorActorRef

Akka Actors

Dispatcher

Mailbox

Threads

ActorActorRef

Akka Actors

Dispatcher

Mailbox

Threads

ActorActorRef

Akka Actors

Dispatcher

Mailbox

Threads

ActorActorRef

One-way messaging

class Act( next : ActorRef ) extends Actor { def receive = { case msg : Msg => { val newMsg = doSomeStuff(msg) next ! newMsg } }}

SharedDispatcher

Two-way

case msg : Msg => { val q = doSomeStuff(msg) next !! q match { case Some(reply) => { //receive reply } case None => .. } }

Q R

case q : Request => { val r = doSomeStuff(q) self.reply_?(q) }

Two-way with Futures

val future = actorRef !!! msg failure { case e: SomeException=> failureResponse}// doesn’t blockfuture.onComplete { f=> // do your response handling… // response can be failureResponse val response = f.get}.onTimeout { => //** Akka 1.2 // do your timeout handling here}

Camel Actors

• Camel Component integration• Consumers and Producers

class Consume(uri:String) extends Actor with Consumer { def endpointUri = uri def receive = { case msg: Message => { // receiving a Camel Message, extract body val data = msg.bodyAs[String] … self.reply("OK") } }}

Camel Consumers

class Consume(uri:String) extends Actor with Consumer { def endpointUri = uri def receive = { case msg: Message => { // receiving a Camel Message, extract body val data = msg.bodyAs[String] … self.reply("OK") } }}

Camel Producers

class Produce(uri:String) extends Actor with Producer { def endpointUri = uri def receiveBeforeProduce = { // you can change the message here before sending case msg => msg }}

Camel Producers

• Use CamelContext

val context = CamelContextManager.context

val ex = context.get.getEndpoint(endpointUri).createExchangeval in = ex.getInin.setHeaders(JavaConversions.mapAsJavaMap(headers))in.setBody(body)ex.setIn(in)CamelContextManager.template.get.asyncCallback(endpointUri, ex, logOnCompetion)

Dispatchers

class Endpoint(dispatcher: MessageDispatcher = Dispatchers. globalExecutorBasedEventDrivenDispatcher) extends Actor { self.dispatcher = dispatcher ….}

ThreadBasedDispatcher

ExecutorBasedEventDrivenDispatcher

ExecutorBasedWorkStealingDispatcher

PriorityExecutorBasedEventDrivenDispatcher

RollYourOwn

Dispatchers

Local Processing Camel ConsumersCamel Producers

Heartbeat,Cleanup,Schedule,Long Running,Redelivery,Remote Error Handling

Remote Actors

Messages

• Case classes• Simple data containers• toProtobuf and fromProtobuf on case class

and companion object for remoting

• Every msg has a • correlation Id to the event• Timestamp

• Logging, Tracing, Performance

Event Process Chains

Load Balancer

Forwarding Actor

EndPoint (Consumer/Producer)

Remote Actors

• Client• Actor.remote.actorFor(name, host,port)

• Server• Actor.remote.register(name, actorRef)

• remoteActorFor: (String, String, Int) => ActorRef = remote.actorFor

Remote Actors Internals

• RemoteActorRef• RemoteClient• Netty Channels• Serialization (Scala,

Java, (S)JSON, ProtoBuf

• RemoteServer• Netty Server• Lifecycle Event

listeners• RemoteProtocol

ActorRef Mailbox ActorRemoteActorRef

RemoteClient RemoteServer

RemoteProtocol

Remote Actors (Akka 1.1.3)

• JBoss Netty (Channels)• Not as transparent as you would like• Remote Lifecycle Listeners• Closing sockets, reconnect• Own Guaranteed Delivery implementation

– Specific requirements– Bases on Idempotent Receiver and Repeating

messages after reconnect– Non-Trivial– Heartbeats– Exp Backoff

Remote configuration

• #compression-scheme = "zlib“ #leave out• zlib-compression-level = 0• client: reconnection-time-window = 2000000• client: read-timeout (we use 60)• backlog = 4096 # Netty backlog for

connections, should suffice, increase if failures happen

Monitoring

• Application level monitoring• Listener Actors contain Custom JMX MBeans• Direct passthrough of msg or translation

class SomeActor(listener: Option[ActorRef] = None) extends Actor { def receive = { case msg:Msg=> { //handle the msg … // passthrough msg listener.foreach { a => a ! msg } } …

class SystemMonitor(reg: BeanRegistry) extends Actor { def receive = { case msg: SomeInterestingEvent => someJMXBean.processMessage(msg) }}

Testing with TestKit

class TestKitUsageSpec extends WordSpec with BeforeAndAfterAll with ShouldMatchers with TestKit { override protected def afterAll(): scala.Unit = { stopTestActor echoRef.stop} "An EchoActor" should { "Respond with the same message it receives" in { within(100 millis) { echoRef ! "test" expectMsg("test") } } }

Testing one-way Actor Output

trait ReplyAfterProcessing extends Actor { abstract override def receive = { super.receive andThen { case msg => self.reply_?(msg) } }}

class SomeSpec extends WordSpec { "A oneway actor" should { "Tell me when it's finished "in { val test = Actor.actorOf(new SomeActor() with ReplyAfterProcessing) var reply = test !! (message, 1000) if(reply.isEmpty) fail("some message") } }}

Microkernel deployment

• sbt dist (akka-sbt-plugin)• Creates dist directory• init.d scripts

Questions?