event sourcing using akka on aws

EVENT‐DRIVEN‐ARCHITECTURE ONAWS USING AKKA

Software Engineer @firstbird

Contact me: @pfeiffer_d_

DANIEL PFEIFFER

THE JOURNEY OF EVENTSthat are stored with Akka Persistenceto be then distributed via AWSto be consumed with Akka Streamwhat comes after that

THE EXAMPLE

DOMAIN DRIVEN DESIGNWe are talking about Aggregates and Events

EVENT SOURCING"Capture all changes to an application

state as a sequence of events" -Martin Fowler, 2005

CQRS"Asking a question should not changethe answer." - Bertrand Meyer, 2012

THE DIFFERENCE

PERSISTENTACTORclass TimeEntryAggregate(id: String) extends PersistentActor{ override def persistenceId: String = ??? override def receiveCommand: Receive = ??? override def receiveRecover: Receive = ???}

A PersistentActor class represents one DDDaggregate.

class TimeEntryAggregate(id: String) extends PersistentActor{ ... override def persistenceId: String = s"time-entry-$id" ...}

A PersistentActor receives commands andpersists events.

case class CreateTimeEntry( begin: DateTime, end: DateTime)

case class TimeEntryCreated( begin: DateTime, end: DateTime, sequenceNr: Long)

A PersistentActor receives commands andpersists events.

class TimeEntryAggregate(id: String) extends PersistentActor{ ... override def receiveCommand: Receive = case CreateTimeEntry(start, end) => val event = TimeEntryCreated(start,end, lastSequenceNr + 1) persist(event){e => // update internal state // here we go after the event is persisted } } ...}

A PersistentActor recovers from the journalclass TimeEntryAggregate(id: String) extends PersistentActor{ ... override def receiveRecover: Receive = { case TimeEntryCreated(start,end) => // update internal state } ...}

STORING EVENTS TO A JOURNALIS NICE, BUT ...

others may be as well interested, so we have todistribute them.

WITHIN ONE JVM WE COULD...use Akka EventStream to publish events ...

class TimeEntryActor() extends PersistentActor{ ... persist(event){e => context.system.eventStream.publish(e) } ...}

... and subscribe from interested actorsclass EventConsumer extends Actor{ override def preStart: Unit = { context.system.eventStream.subscribe(classOf[TimeEntryActor.TimeEntryCreated } override def receive: Receive = { case e: TimeEntryCreated => //do something with that event }}

HOW DO I INTERACT WITH MYAGGREGATES?

DON'T CALL ME! CALL MYOFFICE!

EXAMPLE 1

BUT WE WANT TO BE COOL

DISTRIBUTED SYSTEMS AREHARD

WE WANT TO MAKE THEMEASIER

INTRODUCE A MESSAGEBROKER, PUB/SUB ...

THE EXAMPLE ONAWS STEROIDS

TIME ENTRY SERVICE

EMAIL SERVICE

SQS FLOWSqsSource(...) .map(msg => Envelope(msg.receiptHandle, msg.body)) .via(unmarshal()) .via(process()) .runWith(ack(...))

SQS MESSAGE{ "messageId" : "", "receiptHandle" : "", "md5OfBody" : "", "body" : ""}

SNS NOTIFICATION ENVELOPE{ "Type" : "Notification", "MessageId" : "", "TopicArn" : "", "Subject" : "time_entry.approved", "Message" : "", "Timestamp" : "", "SignatureVersion" : "", "Signature" : "", "SigningCertURL" : "", "UnsubscribeURL" : ""}

CHALLENGES

EVENT SCHEMA EVOLUTIONadding a field to an event type,remove or rename field in event type,remove event type,split event into multiple smaller events.

case class TimeEntryCreated( id: UUID, begin: DateTime, end: DateTime, timeEntryUserId: UUID, userId: UUID)

gets marshalled to{ "id" : "123456", "begin" : "2016-09-01 12:00:00", "end" : "2016-09-01 12:15:00", "timeEntryUserId" : "00000000-0000-0000-0000-000000000000", "userId" : "00000000-0000-0000-0000-000000000000"}

Then something within our schema changescase class TimeEntryCreated( ... description: String ...)

and we will have fun with that one{ "id" : "123456", "begin" : "2016-09-01 12:00:00", "end" : "2016-09-01 12:15:00", "timeEntryUserId" : "00000000-0000-0000-0000-000000000000", "userId" : "00000000-0000-0000-0000-000000000000"}

{ "id" : "123456", "begin" : "2016-09-01 12:00:00", "end" : "2016-09-01 12:15:00", "timeEntryUserId" : "00000000-0000-0000-0000-000000000000", "userId" : "00000000-0000-0000-0000-000000000000"}

needs to be transformed to that{ "id" : "123456", "begin" : "2016-09-01 12:00:00", "end" : "2016-09-01 12:15:00", "timeEntryUserId" : "00000000-0000-0000-0000-000000000000", "userId" : "00000000-0000-0000-0000-000000000000", "description" : "N/A"}

class Evolution(system: ExtendedActorSystem) extends EventAdapter{} override def fromJournal(event: Any, manifest: String): EventSeq = { val es = ... //doing evolution on event EventSeq(es) } override def manifest(event: Any): String = { val version = ??? event.getClass.getSimpleName + ":" + version } override def toJournal(event: Any): Any = ??? //write to the journal}

SCALING WITH AKKA CLUSTERON AWS

or "Who am I, and where are all the others?"

JOINING A CLUSTER

gives you information about your instancehttp://169.254.169.254/latest/meta-data/instance-id

PRESERVING MESSAGE ORDER

The query side could look like that| id | ... | version || --- | --- | ------- || 1 | ... | 45 || 2 | ... | 3 || 3 | ... | 58 |

CHOOSING THE RIGHTDATASTORE

you will make mistakes so plan for it

ASYNCHRONOUS MEANS NOTIMMEDIATE

WHY DO WE REALLY WANT TODO THAT?

WE WANT TO SLEEP BETTER

WE WANT TO GET RID OF OURMONOLITH

OUR DEVELOPMENT PROCESS BENEFITS

WE CAN ALWAYS AGGREGATE OUREVENTS

IF SOMEONE STARTS TO ASKQUESTIONS...

An audit log is included in your application for free!

SUMMARY

THE FIRST PROTOTYPE IS EASY BUT TOTACKLE THE CHALLENGES NEEDS

EFFORT!

CONFIGURE YOUR SQS QUEUESPROPERLY

LET EACH SERVICE MANAGE IT'SRESOURCES ITSELF.

ONE TOPIC PER AGGREGATE

LITERATURE

https://www.infoq.com/articles/AmazonPubSubhttp://martinfowler.com/eaaDev/EventCollaboration.html

http://martinfowler.com/bliki/CQRS.htmlhttps://github.com/dpfeiffer/event-sourcing-aws-

akka-showcasehttp://www.oreilly.com/programming/free/reactive-

microservices-architecture.htmlhttp://doc.akka.io/docs/akka/snapshot/scala/persistence-

schema-evolution.html#persistence-schema-evolution-scala http://chrisloy.net/2014/05/11/akka-

cluster-ec2-autoscaling.html