voxxed bristol 2017 - from c to q, one event at a time: event sourcing illustrated by lorenzo nicora

Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX

From C to Q, one Event at a time

1

Lorenzo Nicora OpenCredo

Event Sourcing illustrated

Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX

Lorenzo Nicora @nicusX

lorenzo.nicora@opencredo.com https://opencredo.com/author/lorenzo/

Senior Consultant @ OpenCredo • Microservices • Cloud • Event-driven, Event Sourcing, Reactive • Java, Spring, Akka…

2

3

Concepts from DDD

Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX

Aggregate

Event Sourcing,CQRS => DDD/

State: f(t)

Domain Event

Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 4

Once upon a time…

Everything was Synchronous

Request <-> Response

Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 5

Scaling out…

Updates —> Locks —> Contention!<— Block! <—

-> Distributed still Synchronous

Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 6

Let’s go Asynchronous

-> Distributed & Message-drivenRequest/Response ACID Transactions

• Distributed: Propagation takes time • Async: No global order strictly guaranteed

Updates applied in the wrong order => Inconsistent state!

Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 7

Command Sourcing💡

• Append Only —> No Contention, No Update • Build State from Commands history

K/V Store Distributed

Command “Submit this Order! (Please)” ->A request (imperative sentence) ->May fail, be rejected ->May affect multiple Aggregates

8

Commands

Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX

✉

Rebuild Aggregate State from Commands

9

Events

Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX

Event “Order submitted” -> Statement of facts (past tense) ->Never fails ->Can’t be changed ->May be designed to affect a single Aggregate

✉

Rebuild Aggregate State from Events

Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 10

Command > Event Sourcing💡

Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 11

Commands to Events

X

Y

Z?

Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 12

(Business) Consistency

(ACID) Transactions

Eventual (Business) Consistency

Guess —> Compensate —> Apologies

Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 13

e.g. Saga

StatefulOut of band

Corrective Action(Command / Event)

Saga

Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX

–Greg Young

“Accountants don’t use pencils.

They use pens”

Corrective Actions

14

Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 15

Benefits of Event Sourcing

History (for free)

Rebuild State at a point in Time

Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 16

Benefits of Event Sourcing

Easier Eventual Business Consistency

—> Corrective Events

Robust to data corruption (bugs, malicious, fat fingers…)

Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 17

Benefits of Event Sourcing

Horizontal Scalability &

Low Latency writes

-> Distributed systems & data store

—> Append-only Log

—> Asynchronous processing

Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 18

Commands to Events

Stateful: Events depends 0n current State

Stateless: Validate, split by Aggregate…

Depends on Business Domain

x Point of synchronisation? Out-of-order commands (IoT, Mobile)

Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 19

Thinking fast or slow

Stateless: Think fast,

Write fast, More thinking later

Stateful: Think slow…(rebuild state),

Write fast, Less thinking later

Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX

What about reads?

20

Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 21

Retrieving the State

How do I retrieve the State? “Get details of Order ‘AB123’”

❔

not very efficient, but… …may work

Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 22

Querying (Searching) the State❓How do query the State?

“Get all Orders delivered to ‘SE1 0NZ’”

Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 23

CQRS

Command Query Responsibility Segregation

💡

Separate • Code • muService • Datastore

-> Update -—> Retrieve

Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 24

Not a new idea

Specialised

Downstream

Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 25

CQRS and Event Sourcing

for low-latenc

y writes

Event Sourcing => CQRS

Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 26

Materialised Views

• In MemoryK/V Store Graph DB RDBMS

• Rebuildable from Events

💡a.k.a. Read-optimised Views Read Views Projections…

Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 27

Materialised Views

* Views are optimised for specific query use cases

—> multiple Views from same Events

* May be updated asynchronously + low latency writes, scalability - delayed + may reorder Events

Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 28

Materialised Views

The Event Log is your Source of Truth

* Easy to evolve or fix —> change or fix logic; rebuild view from events

Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 29

Indexes

• Search EnginesK/V Stores Graph DB

+ Optimised for querying (less for retrieving) + Reduced delay of State

💡

Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 30

Hybrid solutions: e.g. Snapshots

+ Speed up rebuilding the State

+ Use recent Events to rebuild up-to-date

💡Long delayed

Distributed + Asynchronous system Distributed ACID Transactions (2PC)

31

No Global Current State

Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX

AS

SA

B

Information propagates at a finite speed

Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 32

Lessonfrom the Trenches

Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 33

Lesson Learned #1

If you put data in…

…you will eventuallyhave to get them out!

The “Query” side is not secondary

Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 34

Lessons Learned #2

In old days:normalising one DB

to support as many queries as possible

With CQRS (also No SQL in general)

multiple denormalised “data stores” optimised for different queries

Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 35

Lessons Learned #3

Centralised Event Store= Event-sourced monolith

Each microservice owns its state (Event Store)

Microservices:

Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX

+++ Summing up +++

36

Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 37

ES/CQRS Optimal Use Cases

• High Volume, Low Latency writes

• Out-of-order Commands (IoT, Mobile)

Event Sourcing + CQRS😋

Domain: stateless Command-> Event

Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 38

ES/CQRS Drawbacks

x No “One-Size-Fits-All”

—> Multiple “Q” implementations

x Delayed reads

x No ACID Transactions

x Additional complexity (!!!)

🙁

Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX 39

ES/CQRS Benefits

+ No “One-Size-Fits-All” —> “Q” optimised per use cases

+ Distributed systems (Microservices)

+ Eventual (Business) Consistency

+ History, Temporal queries

+ Robust to data corruption

😀

Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX

That’s all, Folks!

40

Voxxed Days Bristol 2017 Lorenzo Nicora - @nicusX

??? Questions ???

41

Thanks.

⏳