resilient applications with akka persistence - scaladays 2014
Post on 27-Aug-2014
5.242 Views
Preview:
DESCRIPTION
TRANSCRIPT
Resilient Applications with Akka Persistence
Patrik Nordwall@patriknw
Konrad Malawski@ktosopl
Björn Antonsson@bantonsson
Reactive Applications
Akka Persistence ScalaDays 2014
Resilient
• Embrace Failure • Failure is a normal part of the application lifecycle
• Self Heal • Failure is detected, isolated, and managed
Akka Persistence ScalaDays 2014
The Naïve Way
• Write State to Database
• Transactions Everywhere
• Problem Solved?
• Not Scalable, Responsive, Event-Driven!
Akka Persistence ScalaDays 2014
Command and Event Sourcing
Command and Event Sourcing
• State is the sum of Events
• Events are persisted to Store
• Append only
• Scales well
Akka Persistence ScalaDays 2014
Command v.s. Event
• Command
• What someone wants me to do
• Can be rejected
• Event
• Something that has already happened
• An immutable fact
Akka Persistence ScalaDays 2014
Commands can Generate Events
• If I accept a Command and change State
• Persist Event to Store
• If I crash
• Replay Events to recover State
Akka Persistence ScalaDays 2014
Persist All Commands?
• If I crash on a Command
• I will likely crash during recovery
• Like the Army
• Don't question orders
• Repeat until success
Akka Persistence ScalaDays 2014
Only Persist Events
• Only accepted Commands generate Events
• No surprises during recovery
• Like a dieting method
• You are what you eat
Akka Persistence ScalaDays 2014
Achievement Unlocked?
• Resilient
• State is recoverable
• Scalable
• Append only writes
• Something Missing?
• Queries
Akka Persistence ScalaDays 2014
CQRSCommand Query Responsibility Segregation
CQRS
• Separate Models
• Command Model
• Optimized for command processing
• Query Model
• Optimized data presentation
Akka Persistence ScalaDays 2014
Query Model from Events
• Source the Events
• Pick what fits
• In Memory
• SQL Database
• Graph Database
• Key Value Store
Akka Persistence ScalaDays 2014
Akka Persistence ScalaDays 2014
Client
Service
Query Model
Command Store
Query Store
Command Model
PersistentActor
Akka Persistence ScalaDays 2014
PersistentActor
Processor & Eventsourced ProcessorReplaces:
in Akka 2.3.4+
super quick domain modelling!
sealed trait Command!case class GiveMe(coins: Int) extends Command!case class TakeMy(coins: Int) extends Command
Commands - what others “tell” us; not persisted
case class Wallet(coins: Int) {! def updated(diff: Int) = State(coins + diff)!}
State - reflection of a series of events
sealed trait Event!case class BalanceChangedBy(coins: Int) extends Event!
Events - reflect effects, past tense; persisted
var state = S0 !
def processorId = “a” !
PersistentActor
Command
!
!
Journal
PersistentActor
var state = S0 !
def processorId = “a” !
!
!
Journal
Generate Events
PersistentActor
var state = S0 !
def processorId = “a” !
!
!
Journal
Generate Events
E1
PersistentActor
ACK “persisted”
!
!
Journal
E1
var state = S0 !
def processorId = “a” !
PersistentActor
“Apply” event
!
!
Journal
E1
var state = S1 !
def processorId = “a” !
E1
PersistentActor
!
!
Journal
E1
var state = S1 !
def processorId = “a” !
E1
Okey!
PersistentActor
!
!
Journal
E1
var state = S1 !
def processorId = “a” !
E1
Okey!
PersistentActor
!
!
Journal
E1
var state = S1 !
def processorId = “a” !
E1
Ok, he got my $.
PersistentActor
class BitCoinWallet extends PersistentActor {!! var state = Wallet(coins = 0)!! def updateState(e: Event): State = {! case BalanceChangedBy(coins) => state.updatedWith(coins)! }! ! // API:!! def receiveCommand = ??? // TODO!! def receiveRecover = ??? // TODO!!}!
persist(e) { e => }
PersistentActor
def receiveCommand = {!! case TakeMy(coins) =>! persist(BalanceChangedBy(coins)) { changed =>! state = updateState(changed) ! }!!!!!!!}
async callback
PersistentActor: persist(){}
def receiveCommand = {!!!!!!! case GiveMe(coins) if coins <= state.coins =>! persist(BalanceChangedBy(-coins)) { changed =>! state = updateState(changed) ! sender() ! TakeMy(coins)! }!}
async callbackSafe to mutate
the Actor’s state
PersistentActor
def receiveCommand = {!!!!!!! case GiveMe(coins) if coins <= state.coins =>! persist(BalanceChangedBy(-coins)) { changed =>! state = updateState(changed) ! sender() ! TakeMy(coins)! }!}
Safe to access sender here
persist(){} - Ordering guarantees
!
!
Journal
E1
var state = S0 !
def processorId = “a” !
C1C2
C3
!
!
Journal
E1
var state = S0 !
def processorId = “a” !
C1C2
C3
Commands get “stashed” until processing C1’s events are acted upon.
persist(){} - Ordering guarantees
!
!
Journal
var state = S0 !
def processorId = “a” !
C1C2
C3 E1
E2
E2E1
events get applied in-order
persist(){} - Ordering guarantees
C2
!
!
Journal
var state = S0 !
def processorId = “a” !
C3 E1 E2
E2E1
and the cycle repeats
persist(){} - Ordering guarantees
persistAsync(e) { e => }
persistAsync(e) { e => } + defer(e) { e => }
def receiveCommand = {!!!! case Mark(id) =>! sender() ! InitMarking! persistAsync(Marker) { m =>! // update state...! }!!!!!}
persistAsync
PersistentActor: persistAsync(){}
will NOT force stashing of commands
PersistentActor: persistAsync(){}
def receiveCommand = {!!!! case Mark(id) =>! sender() ! InitMarking! persistAsync(Marker) { m =>! // update state...! }!! defer(Marked(id)) { marked =>! sender() ! marked! }!}
execute once all persistAsync handlers done
NOT persisted
persistAsync(){} - Ordering guarantees
!
!
Journal
var state = S0 !
def processorId = “a” !
C1C2
C3
persistAsync(){} - Ordering guarantees
!
!
Journal
var state = S0 !
def processorId = “a” !C2
C3
persistAsync(){} - Ordering guarantees
!
!
Journal
var state = S0 !
def processorId = “a” !
C3
persistAsync(){} - Ordering guarantees
!
!
Journal
var state = S0 !
def processorId = “a” !
C3
E1
E2
persistAsync(){} - Ordering guarantees
var state = S0 !
def processorId = “a” !
C3
E1
Akka Persistence ScalaDays
!
!
Journal
E1
E2
persistAsync(){} - Ordering guarantees
E1
var state = S1 !
def processorId = “a” !
E2
E1
E2
!
!
JournalAkka Persistence ScalaDays
E2
E3E1
persistAsync(){} - Ordering guarantees
E1
var state = S2 !
def processorId = “a” !
E2
E1 E2
deferred handlers triggered
M1M2
!
!
JournalAkka Persistence ScalaDays
E2
E3E1
Recovery
Akka Persistence ScalaDays
Eventsourced, recovery
/** MUST NOT SIDE-EFFECT! */!def receiveRecover = {! case replayedEvent: Event => ! state = updateState(replayedEvent)!}
re-using updateState, as seen in receiveCommand
Akka Persistence ScalaDays
Views
Akka Persistence ScalaDays
Journal (DB)
!
!
!
Views
!Processor
!def processorId = “a”
!
polling
Akka Persistence ScalaDays
!View
!def processorId = “a”
!!!
Journal (DB)
!
!
!
Views
!Processor
!def processorId = “a”
!
polling
!View
!def processorId = “a”
!!!
polling
different ActorPath, same processorId
Akka Persistence ScalaDays
!View
!def processorId = “a”
!!!
View
class DoublingCounterProcessor extends View {! var state = 0! override val processorId = "counter"!! def receive = {! case Persistent(payload, seqNr) =>! // “state += 2 * payload” !! }!}
subject to change!
Akka Persistence ScalaDays
Views, as Reactive Streams
Akka Persistence ScalaDays
View, as ReactiveStream
// Imports ...!!import org.reactivestreams.api.Producer!!import akka.stream._!import akka.stream.scaladsl.Flow!!import akka.persistence._!import akka.persistence.stream._!
val materializer = FlowMaterializer(MaterializerSettings())!
pull request by krasserm
early preview
Akka Persistence ScalaDays
View, as ReactiveStream
// 1 producer and 2 consumers:!val p1: Producer[Persistent] = PersistentFlow.! fromProcessor(“processor-1").! toProducer(materializer)!!Flow(p1).! foreach(p => println(s"consumer-1: ${p.payload}”)).! consume(materializer)!!Flow(p1).! foreach(p => println(s"consumer-2: ${p.payload}”)).! consume(materializer)
pull request by krasserm
early preview
Akka Persistence ScalaDays
View, as ReactiveStream
// 2 producers (merged) and 1 consumer:!val p2: Producer[Persistent] = PersistentFlow.! fromProcessor(“processor-2").! toProducer(materializer)!val p3: Producer[Persistent] = PersistentFlow.! fromProcessor(“processor-3").! toProducer(materializer)!!Flow(p2).merge(p3). // triggers on “either”! foreach { p => println(s"consumer-3: ${p.payload}") }.! consume(materializer)!
pull request by krasserm
early preview
Akka Persistence ScalaDays
Akka Persistence ScalaDays 2014
Usage in a Cluster
• distributed journal (http://akka.io/community/)
• Cassandra
• DynamoDB
• HBase
• MongoDB
• shared LevelDB journal for testing
• single writer
• cluster singleton
• cluster sharding
Akka Persistence ScalaDays 2014
Cluster Singleton
AB
C D
Akka Persistence ScalaDays 2014
Cluster Singleton
AB
C
D
role: backend-1 role: backend-1
role: backend-2 role: backend-2
Akka Persistence ScalaDays 2014
Cluster Sharding
A B
C D
Akka Persistence ScalaDays 2014
Cluster Sharding
sender
id:17
region node-‐1
coordinator
region node-‐2
region node-‐3
GetShardHome:17
id:17 ShardHome:17 -‐> node2
17 -‐> node2
Akka Persistence ScalaDays 2014
Cluster Sharding
sender region node-‐1
coordinator
region node-‐2
region node-‐3
id:17
id:17GetShardHome:17
ShardHome:17 -‐> node2
id:17
17 -‐> node2
17 -‐> node2
Akka Persistence ScalaDays 2014
Cluster Sharding
17
sender region node-‐1
coordinator
region node-‐2
region node-‐3
id:17
id:17
17 -‐> node2
17 -‐> node2
17 -‐> node2
Akka Persistence ScalaDays 2014
Cluster Sharding
17
sender region node-‐1
coordinator
region node-‐2
region node-‐3
17 -‐> node2
17 -‐> node2
17 -‐> node2
id:17
Akka Persistence ScalaDays 2014
Cluster Sharding
17
sender region node-‐1
coordinator
region node-‐2
region node-‐3
17 -‐> node2
17 -‐> node2
17 -‐> node2
id:17
Cluster Sharding
val idExtractor: ShardRegion.IdExtractor = { case cmd: Command => (cmd.postId, cmd) } ! val shardResolver: ShardRegion.ShardResolver = msg => msg match { case cmd: Command => (math.abs(cmd.postId.hashCode) % 100).toString }
ClusterSharding(system).start( typeName = BlogPost.shardName, entryProps = Some(BlogPost.props()), idExtractor = BlogPost.idExtractor, shardResolver = BlogPost.shardResolver)
val blogPostRegion: ActorRef = ClusterSharding(context.system).shardRegion(BlogPost.shardName) !val postId = UUID.randomUUID().toString blogPostRegion ! BlogPost.AddPost(postId, author, title)
Akka Persistence ScalaDays 2014
Lost messages
sender destination
$
Akka Persistence ScalaDays 2014
At-least-once delivery - duplicates
sender destination
$
ok
$
$$
ok
Re-‐send
Akka Persistence ScalaDays 2014
M2
At-least-once delivery - unordered
sender destination
M1
ok 1 ok 2
M2
ok 3
M3
M1M3M2
Re-‐send
Akka Persistence ScalaDays 2014
M2
At-least-once delivery - crash
sender destination
M1
ok 1 ok 2
M2
ok 3
M3
1. Sent M1 2. Sent M2 3. Sent M3
M3
5. M2 Confirmed 6. M3 Confirmed
4. M1 Confirmed
senderM1M2
M3
PersistentActor with AtLeastOnceDelivery
case class Msg(deliveryId: Long, s: String) case class Confirm(deliveryId: Long) sealed trait Evt case class MsgSent(s: String) extends Evt case class MsgConfirmed(deliveryId: Long) extends Evt
class Sender(destination: ActorPath) extends PersistentActor with AtLeastOnceDelivery { ! def receiveCommand: Receive = { case s: String => persist(MsgSent(s))(updateState) case Confirm(deliveryId) => persist(MsgConfirmed(deliveryId))(updateState) } ! def receiveRecover: Receive = { case evt: Evt => updateState(evt) } ! def updateState(evt: Evt): Unit = evt match { case MsgSent(s) => deliver(destination, deliveryId => Msg(deliveryId, s)) ! case MsgConfirmed(deliveryId) => confirmDelivery(deliveryId) } }
Akka Persistence ScalaDays 2014
Next step
• Documentation • http://doc.akka.io/docs/akka/2.3.3/scala/persistence.html
• http://doc.akka.io/docs/akka/2.3.3/java/persistence.html
• http://doc.akka.io/docs/akka/2.3.3/contrib/cluster-sharding.html
• Typesafe Activator • https://typesafe.com/activator/template/akka-sample-persistence-scala
• https://typesafe.com/activator/template/akka-sample-persistence-java
• http://typesafe.com/activator/template/akka-cluster-sharding-scala
• Mailing list • http://groups.google.com/group/akka-user
• Migration guide from Eventsourced • http://doc.akka.io/docs/akka/2.3.3/project/migration-guide-eventsourced-2.3.x.html
©Typesafe 2014 – All Rights Reserved
top related