akka persistence scalar conf 2014 140404160618 phpapp02
DESCRIPTION
akka persistenceTRANSCRIPT
-
Akka persistence
(message sourcing in 30 minutes)
Konrad 'ktoso' Malawski
Scalar 2014 @ Warsaw, PL
-
Konrad `@ktosopl` Malawski
typesafe.com
geecon.org
Java.pl / KrakowScala.pl
sckrk.com / meetup.com/Paper-Cup @ London
GDGKrakow.pl
meetup.com/Lambda-Lounge-Krakow
-
Konrad `@ktosopl` Malawski
typesafe.com
geecon.org
Java.pl / KrakowScala.pl
sckrk.com / meetup.com/Paper-Cup @ London
GDGKrakow.pl
meetup.com/Lambda-Lounge-Krakow
-
Konrad `@ktosopl` Malawski
typesafe.com
geecon.org
Java.pl / KrakowScala.pl
sckrk.com / meetup.com/Paper-Cup @ London
GDGKrakow.pl
meetup.com/Lambda-Lounge-Krakow
-
Konrad `@ktosopl` Malawski
typesafe.com
geecon.org
Java.pl / KrakowScala.pl
sckrk.com / meetup.com/Paper-Cup @ London
GDGKrakow.pl
meetup.com/Lambda-Lounge-Krakow
-
Konrad `@ktosopl` Malawski
typesafe.com
geecon.org
Java.pl / KrakowScala.pl
sckrk.com / meetup.com/Paper-Cup @ London
GDGKrakow.pl
meetup.com/Lambda-Lounge-Krakow
-
Konrad `@ktosopl` Malawski
typesafe.com
geecon.org
Java.pl / KrakowScala.pl
sckrk.com / meetup.com/Paper-Cup @ London
GDGKrakow.pl
meetup.com/Lambda-Lounge-Krakow
hAkker @
-
mainly by
Martin Krasser !!(as contractor for Typesafe)
!inspired by:
akka-persistence
https://github.com/krasserm
https://github.com/eligosource/eventsourced
-
dependencies
libraryDependencies ++= Seq(! "com.typesafe.akka" %% akka-actor" % "2.3.0",! "com.typesafe.akka" %% "akka-persistence-experimental" % "2.3.0"!)
-
Show of hands!
-
Show of hands!
-
Show of hands!
-
Show of hands!
-
sourcing styles
Command Sourcing Event Sourcing
msg: DoThing
msg persisted before receive
imperative, do the thing
business logic change, can be reflected in reaction
Processor
-
sourcing styles
Command Sourcing Event Sourcing
msg: DoThing msg: ThingDone
msg persisted before receive commands converted to events, must be manually persistedimperative, do the thing past tense, happened
business logic change, can be reflected in reaction
business logic change, wont change previous events
Processor EventsourcedProcessor
-
Plain Actors
-
count: 0 !
!
Actor
-
count: 0 !
!
ActorAn Actor that keeps count of messages it processed
-
count: 0 !
!
ActorAn Actor that keeps count of messages it processed
-
count: 0 !
!
ActorAn Actor that keeps count of messages it processed
Lets send 2 messages to it
-
count: 0 !
!
ActorAn Actor that keeps count of messages it processed
Lets send 2 messages to it(its commands)
-
Actor
!!class Counter extends Actor {! var count = 0! def receive = {! case _ => count += 1! }!}
-
count: 0 !
!
Actor
-
count: 0 !
!
Actor
-
count: 1 !
!
Actor
-
count: 1 !
!
Actor
crash!
-
Actor
crash!
-
Actor
restart
-
count: 0 !
!
Actor
restart
-
count: 0 !
!
Actor
restarted
-
count: 1 !
!
Actor
restarted
-
count: 1 !
!
Actor
restarted
-
count: 1 !
!wrong!
expected count == 2!
Actor
restarted
-
Processor
-
var count = 0 !
def processorId = a !
Journal
(DB)
!!!
Processor
-
Journal
(DB)
!!!
Processor
var count = 0 !
def processorId = a !
-
Journal
(DB)
!!!
Processor
var count = 0 !
def processorId = a !
-
Journal
(DB)
!!!
Processor
var count = 0 !
def processorId = a !
-
Journal
(DB)
!!!
Processor
var count = 1 !
def processorId = a !
-
Journal
(DB)
!!!
Processor
var count = 1 !
def processorId = a !
-
Journal
(DB)
!!!
Processor
var count = 1 !
def processorId = a !
crash!
-
Journal
(DB)
!!!
Processor
-
Journal
(DB)
!!!
Processor
restart
-
Journal
(DB)
!!!
Processor
var count = 0 !
def processorId = a !
restart
-
Journal
(DB)
!!!
Processor
var count = 0 !
def processorId = a !
replay!
restart
-
Journal
(DB)
!!!
Processor
var count = 0 !
def processorId = a !
-
Journal
(DB)
!!!
Processor
var count = 0 !
def processorId = a !
replay!
-
Journal
(DB)
!!!
Processor
var count = 1 !
def processorId = a !
-
Journal
(DB)
!!!
Processor
var count = 1 !
def processorId = a !
replay!
-
Journal
(DB)
!!!
Processor
var count = 1 !
def processorId = a !
-
Journal
(DB)
!!!
Processor
var count = 2 !
def processorId = a !
-
Journal
(DB)
!!!
Processor
var count = 2 !
def processorId = a !
yay!
-
Processorimport akka.persistence._!!class CounterProcessor extends Processor {! var count = 0! override val processorId = "counter"!! def receive = {! case Persistent(payload, seqNr) =>! // payload already persisted! count += 1! }!}
-
Processorimport akka.persistence._!!class CounterProcessor extends Processor {! var count = 0! override val processorId = "counter"!! def receive = {! case Persistent(payload, seqNr) =>! // payload already persisted! count += 1! }!}
counter ! Persistent(payload)
-
Processorimport akka.persistence._!!class CounterProcessor extends Processor {! var count = 0! override val processorId = "counter"!! def receive = {! case Persistent(payload, seqNr) =>! // payload already persisted! count += 1! }!}
counter ! Persistent(payload)
-
Processorimport akka.persistence._!!class CounterProcessor extends Processor {! var count = 0! override val processorId = "counter"!! def receive = {! case Persistent(payload, seqNr) =>! // payload already persisted! count += 1! }!}
counter ! Persistent(payload)
-
Processorimport akka.persistence._!!class CounterProcessor extends Processor {! var count = 0! override val processorId = "counter"!! def receive = {! case Persistent(payload, seqNr) =>! // payload already persisted! count += 1! }!}
counter ! Persistent(payload)
is already persisted!
-
Processorimport akka.persistence._!!class CounterProcessor extends Processor {! var count = 0! override val processorId = "counter"!! def receive = {! case Persistent(payload, seqNr) =>! // payload already persisted! count += 1! }!}
counter ! Persistent(payload)
sequenceNr (generated by akka)
is already persisted!
-
Processorimport akka.persistence._!!class CounterProcessor extends Processor {! var count = 0! override val processorId = "counter"!! def receive = {! case notPersisted =>! // will not replay this msg!! count += 1! }!}
counter ! payload
-
Processorimport akka.persistence._!!class CounterProcessor extends Processor {! var count = 0! override val processorId = "counter"!! def receive = {! case notPersisted =>! // will not replay this msg!! count += 1! }!}
counter ! payload
wont persist
-
Processorimport akka.persistence._!!class CounterProcessor extends Processor {! var count = 0! override val processorId = "counter"!! def receive = {! case notPersisted =>! // will not replay this msg!! count += 1! }!}
counter ! payload
wont persist
wont replay
-
Processorimport akka.persistence._!!class CounterProcessor extends Processor {! var count = 0! override val processorId = "counter"!! def receive = {! case Persistent(payload, seqNr) =>! // payload already persisted! count += 1!! case notPersistentMsg =>! // msg not persisted - like in normal Actor! count += 1! }!}
-
ProcessorUpsides
Persistent Command Sourcing out of the box
Pretty simple, persist handled for you
Once you get the msg, its persisted
Pluggable Journals (HBase, Cassandra, Mongo, )
Can replay to given seqNr (post-mortem etc!)
-
ProcessorDownsides
Exposes Persistent() to Actors who talk to you
No room for validation before persisting
Theres one Model, we act on the incoming msg
Lower throughput than plain Actor (limited by DB)
-
EventsourcedProcessor
-
EventsourcedProcessor
(longer name == more flexibility);-)
-
super quick domain modeling!
-
super quick domain modeling!
sealed trait Command!case class ManyCommand(nums: List[Int]) extends Command
Commands - input from user, send emails, not persisted
-
super quick domain modeling!
sealed trait Command!case class ManyCommand(nums: List[Int]) extends Command
Commands - input from user, send emails, not persisted
sealed trait Event!case class AddOneEvent(num: Int) extends Event!
Events - business events emitted by the processor, persisted
-
super quick domain modeling!
sealed trait Command!case class ManyCommand(nums: List[Int]) extends Command
Commands - input from user, send emails, not persisted
case class State(count: Int) {! def updated(more: Int) = State(count + more)!}
State - internal actor state, kept in var
sealed trait Event!case class AddOneEvent(num: Int) extends Event!
Events - business events emitted by the processor, persisted
-
var count = 0 !
def processorId = a !
EventsourcedProcessor
!!
Journal
C1
-
var count = 0 !
def processorId = a !
EventsourcedProcessor
Command
!!
Journal
C1
-
var count = 0 !
def processorId = a !
EventsourcedProcessor
!!
Journal
C1
-
var count = 0 !
def processorId = a !
EventsourcedProcessor
!!
Journal
C1
Generate Events
-
C1
var count = 0 !
def processorId = a !
EventsourcedProcessor
!!
Journal
E1
-
C1
var count = 0 !
def processorId = a !
EventsourcedProcessor
!!
Journal
E1
Event
-
C1
var count = 0 !
def processorId = a !
EventsourcedProcessor
!!
Journal
E1
-
C1
var count = 0 !
def processorId = a !
EventsourcedProcessor
!!
Journal
E1
ACK persisted
-
C1
var count = 1 !
def processorId = a !
EventsourcedProcessor
!!
Journal
E1
E1
-
C1
var count = 1 !
def processorId = a !
EventsourcedProcessor
!!
Journal
E1
E1
Apply event
-
EventsourcedProcessor
class MultiCounter extends EventsourcedProcessor {!! var state = State(count = 0)!! def updateState(e: Event): State = {! case event: AddOneEvent => state.updated(event.num)! }! ! // API:!! def receiveCommand = ??? // TODO!! def receiveRecover = ??? // TODO!!}!
-
EventsourcedProcessor
def receiveCommand = {! case ManyCommand(many) =>! for (event ! persist(AddOneEvent(command)) { state = updateState(_) }!}
-
EventsourcedProcessor
def receiveCommand = {! case ManyCommand(many) =>! for (event ! persist(AddOneEvent(command)) { state = updateState(_) }!}
-
EventsourcedProcessor
def receiveCommand = {! case ManyCommand(many) =>! for (event ! persist(AddOneEvent(command)) { state = updateState(_) }!}
-
EventsourcedProcessor
def receiveCommand = {! case ManyCommand(many) =>! for (event ! persist(AddOneEvent(command)) { state = updateState(_) }!}
async persist that event
-
EventsourcedProcessor
def receiveCommand = {! case ManyCommand(many) =>! for (event ! persist(AddOneEvent(command)) { state = updateState(_) }!}
async persist that event
async called after successful persist
-
EventsourcedProcessor
def receiveCommand = {! case ManyCommand(many) =>! for (event ! persist(AddOneEvent(command)) { state = updateState(_) }!}
async persist that event
async called after successful persist
It is guaranteed that no new commands will be received by a processor between a call to `persist` and the execution of its `handler`.
-
EventsourcedProcessor
def receiveCommand = {! case ManyCommand(many) =>! for (event ! persist(AddOneEvent(command)) { state = updateState(_) }!}
- EventsourcedProcessorcase ManyCommand(many) =>! for (event
- EventsourcedProcessorcase ManyCommand(many) =>! for (event
-
EventsourcedProcessorcase ManyCommand(many) =>! for (event ! persist(many map AddOneEvent) { state = updateState(_) }
style fix
-
Eventsourced, recovery
/** MUST NOT SIDE-EFFECT! */!def receiveRecover = {! case replayedEvent: Event => ! updateState(replayedEvent)!}
-
Eventsourced, snapshots
-
Eventsourced, snapshots/** MUST NOT SIDE-EFFECT! */!def receiveRecover = {! case SnapshotOffer(meta, snapshot: State) => ! this.state = state!! case replayedEvent: Event => ! updateState(replayedEvent)!}
-
Eventsourced, snapshots/** MUST NOT SIDE-EFFECT! */!def receiveRecover = {! case SnapshotOffer(meta, snapshot: State) => ! this.state = state!! case replayedEvent: Event => ! updateState(replayedEvent)!}
-
Eventsourced, snapshots/** MUST NOT SIDE-EFFECT! */!def receiveRecover = {! case SnapshotOffer(meta, snapshot: State) => ! this.state = state!! case replayedEvent: Event => ! updateState(replayedEvent)!}
snapshot!? how?
-
Eventsourced, snapshots
def receiveCommand = {! case command: Command =>! saveSnapshot(state) // async!!}
/** MUST NOT SIDE-EFFECT! */!def receiveRecover = {! case SnapshotOffer(meta, snapshot: State) => ! this.state = state!! case replayedEvent: Event => ! updateState(replayedEvent)!}
snapshot!? how?
-
Snapshots
-
Snapshots
(in SnapshotStore)
-
sum of states
Snapshots
!!
Journal
E1 E2 E3 E4
E5 E6 E7 E8
-
state until [E8]
Snapshots
S8!!
Journal
E1 E2 E3 E4
E5 E6 E7 E8
-
state until [E8]
Snapshots
S8
!!
Snapshot Store
snapshot!
!!
Journal
E1 E2 E3 E4
E5 E6 E7 E8
-
state until [E8]
Snapshots
S8
!!
Snapshot Store
!!
Journal
E1 E2 E3 E4
E5 E6 E7 E8
S8
-
state until [E8]
Snapshots
S8
!!
Snapshot Store
!!
Journal
E1 E2 E3 E4
E5 E6 E7 E8
S8
crash!
-
Snapshots
!!
Snapshot Store
!!
Journal
E1 E2 E3 E4
E5 E6 E7 E8
S8
crash!
-
bring me up-to-date!
Snapshots
!!
Snapshot Store
!!
Journal
E1 E2 E3 E4
E5 E6 E7 E8
S8
restart!
-
bring me up-to-date!
Snapshots
!!
Snapshot Store
!!
Journal
E1 E2 E3 E4
E5 E6 E7 E8
S8
restart!replay!
-
bring me up-to-date!
Snapshots
!!
Snapshot Store
S8
restart!replay!
!!
Journal
E1 E2 E3 E4
E5 E6 E7 E8
-
bring me up-to-date!
Snapshots
!!
Snapshot Store
S8
restart!replay!
!!
Journal
E1 E2 E3 E4
E5 E6 E7 E8
-
bring me up-to-date!
Snapshots
!!
Snapshot Store
S8
restart!replay!
S8!!
Journal
E1 E2 E3 E4
E5 E6 E7 E8
-
state until [E8]
Snapshots
!!
Snapshot Store
S8
restart!replay!
S8!!
Journal
E1 E2 E3 E4
E5 E6 E7 E8
-
state until [E8]
Snapshots
!!
Snapshot Store
S8
S8!!
Journal
E1 E2 E3 E4
E5 E6 E7 E8
-
state until [E8]
Snapshots
!!
Snapshot Store
S8
S8!!
Journal
E1 E2 E3 E4
E5 E6 E7 E8
We could delete these!
-
trait MySummer extends Processor {! var nums: List[Int]! var total: Int! ! def receive = {! case "snap" => saveSnapshot(total)! case SaveSnapshotSuccess(metadata) => // ...! case SaveSnapshotFailure(metadata, reason) => // ...! }!}!
Snapshots, save
-
trait MySummer extends Processor {! var nums: List[Int]! var total: Int! ! def receive = {! case "snap" => saveSnapshot(total)! case SaveSnapshotSuccess(metadata) => // ...! case SaveSnapshotFailure(metadata, reason) => // ...! }!}!
Snapshots, save
Async!
-
trait MySummer extends Processor {! var nums: List[Int]! var total: Int! ! def receive = {! case "snap" => saveSnapshot(total)! case SaveSnapshotSuccess(metadata) => // ...! case SaveSnapshotFailure(metadata, reason) => // ...! }!}!
Snapshots, success
-
trait MySummer extends Processor {! var nums: List[Int]! var total: Int! ! def receive = {! case "snap" => saveSnapshot(total)! case SaveSnapshotSuccess(metadata) => // ...! case SaveSnapshotFailure(metadata, reason) => // ...! }!}!
Snapshots, success
final case class SnapshotMetadata(! processorId: String, sequenceNr: Long, ! timestamp: Long)
-
trait MySummer extends Processor {! var nums: List[Int]! var total: Int! ! def receive = {! case "snap" => saveSnapshot(total)! case SaveSnapshotSuccess(metadata) => // ...! case SaveSnapshotFailure(metadata, reason) => // ...! }!}!
Snapshots, success
-
Snapshot Recovery
class Counter extends Processor {! var total = 0! ! def receive = {! case SnapshotOffer(metadata, snap: Int) => ! total = snap!! case Persistent(payload, sequenceNr) => // ...! }!}
-
SnapshotsUpsides
Simple!
Faster recovery (!)
Allows to delete older events
known state at point in time
-
SnapshotsDownsides
More logic to write
Maybe not needed if events replay fast enough
Possibly yet another database to pick
snapshots are different than events, may be big!
-
Views
-
Journal
(DB)
!!!
Views!
Processor !
def processorId = a !
-
Journal
(DB)
!!!
Views!
Processor !
def processorId = a !
!View
!def processorId = a
!!!
-
Journal
(DB)
!!!
Views!
Processor !
def processorId = a !
!View
!def processorId = a
!!!
pooling
-
Journal
(DB)
!!!
Views!
Processor !
def processorId = a !
!View
!def processorId = a
!!!
pooling
!View
!def processorId = a
!!!
pooling
-
Journal
(DB)
!!!
Views!
Processor !
def processorId = a !
!View
!def processorId = a
!!!
pooling
!View
!def processorId = a
!!!
pooling
different ActorPath, same processorId
-
View
class DoublingCounterProcessor extends View {! var state = 0! override val processorId = "counter"!! def receive = {! case Persistent(payload, seqNr) =>! // state += 2 * payload !! }!}
-
Akka Persistence Plugins
-
Akka Persistence PluginsPlugins are Community maintained!
(not sure how production ready)
http://akka.io/community/#journal_plugins
-
Akka Persistence PluginsPlugins are Community maintained!
(not sure how production ready)
http://akka.io/community/#journal_plugins
Journals - Cassandra, HBase, Mongo
-
Akka Persistence PluginsPlugins are Community maintained!
(not sure how production ready)
http://akka.io/community/#journal_plugins
Journals - Cassandra, HBase, Mongo
SnapshotStores - Cassandra, HDFS, HBase, Mongo
-
SnapshotStore Plugins!
http://akka.io/community/#journal_plugins
-
Links Official docs: http://doc.akka.io/docs/akka/2.3.0/scala/persistence.html
Patriks Slides & Webinar: http://www.slideshare.net/patriknw/akka-
persistence-webinar
Papers:
Sagas http://www.cs.cornell.edu/andru/cs711/2002fa/reading/
sagas.pdf
Life beyond Distributed Transactions: http://www-db.cs.wisc.edu/cidr/cidr2007/papers/cidr07p15.pdf
Pics:
http://misaspuppy.deviantart.com/art/Messenger-s-Cutie-Mark-A-
Flying-Envelope-291040459
-
Mailing List
groups.google.com/forum/#!forum/akka-user
-
LinksEric Evans - the DDD book
http://www.amazon.com/Domain-Driven-Design-Tackling-Complexity-Software/dp/0321125215
!!!!!!
Vaughn Vernons Book and Talk
http://www.amazon.com/Implementing-Domain-Driven-Design-Vaughn-Vernon/dp/0321834577
video: https://skillsmatter.com/skillscasts/4698-reactive-ddd-with-scala-and-akka
!!!
-
ktoso @ typesafe.com twitter: ktosopl
github: ktoso
blog: project13.pl
team blog: letitcrash.com Scalar 2014 @ Warsaw, PL
!
Dziki!
Thanks!
!
!
ping me: