presenters - qcon new york · presenters phillipa avery senior software engineer project technical...

Presenters

Phillipa AverySenior Software Engineer

Project Technical Lead and EngineerDownloads License Accounting

[email protected] / @PhillipaAvery

Robert RetaSenior Software Engineer

Event Sourcing System ArchitectDownloads License Accounting

[email protected] / @rreta04

mailto:[email protected]




Phillipa

Robert

Overview

1. Why do we need a Downloads License Accounting Service?

2. Event Sourcing overview.3. Deep dive into the Event Sourcing Architecture.4. What’s it like working with the License Accounting

Service after release?

LICENSE

CREATE PLAYBACK CONTEXT

SESSION EVENTS(START, PAUSE,

RESUME, KEEPALIVE)

Streaming Playback Lifecycle

RELEASELICENSE

SESSION EVENT(STOP)

STOP

Streaming Playback Lifecycle

LICENSE

CREATE PLAYBACK CONTEXT

DOWNLOADED ENCODED MOVIE

Downloads Playback Lifecycle

~ 1 year

SESSION EVENTS(START, PAUSE, RESUME,

KEEPALIVE, STOP)

RENEWLICENSE?

X

RELEASELICENSE

Downloads Playback Lifecycle

Download Business Requirements

● Devices with downloads


● Devices with downloads● Downloads per studio


● Devices with downloads● Downloads per studio● Movie downloads or playback over year

~ 1 year

License Accounting Service Requirements

❏ Flexible

❏ Debuggable

❏ Reliable

❏ Scalable

Flexibility

Debuggability

Scalability

Reliability

License Accounting Service Requirements...

✓ Flexible: data model can be changed❏ RDBMS❏ Document Model



✓ Debuggable❏ Event Sourcing




✓ Reliable❏ Fallbacks




✓ Reliable❏ Fallbacks

✓ Scalable

Event Sourcing

Domain Model

Make Purple

Command

Commands

Commands

Make Purple

Command

Command Handler

Added Red Event

Added Blue Event

Make Purple

Command

Command Handler

Commands

Events

Added Red Event

Added Blue Event

Event Handler

Events

Added Blue Event

Event Handler

Domain Model

Aggregate

Commands

Make Red Command

Command Handler

Command Handler

Commands

Make Red Command

Aggregate ServiceREST

Endpoint

Aggregate Repository

Event StoreQuery Query Query

Event Store

Row ID Events

Row ID Events

e1 e2 e3 e5 e6 e7e4

Event Store

Aggregate id 1 Aggregate id 2


Query Event Store


Event StoreQuery

select * from eventswhere rowId =

Aggregate Repository Event StoreResponse

e1 e2 e3 e5 e6 e7e4


e1 e2 e3 e5 e6 e7e4


e1 e2 e5 e6 e3 e4 e7


Aggregate id 2

Aggregate id 1

e1 e2 e5 e6

e3 e4 e7

Aggregate 1

Aggregate 2


Aggregate 2

Aggregate 1

Aggregate id 2

Aggregate id 1

e1e2 e5 e6

e3 e4 e7


Aggregate 2

Aggregate 1

Aggregate id 2

Aggregate id 1

e2e5 e6

e3 e7


Aggregate 2

Aggregate 1

Aggregate id 2

Aggregate id 1

e5e6

e7


Aggregate 2

Aggregate 1

Aggregate id 2

Aggregate id 1

e6


Aggregate 2

Aggregate 1


Command

Aggregate 1

Command Handler


Command Handler

e8 e9


e8 e9Aggregate 1


Aggregate 1e8 e9


Aggregate 1e9


Aggregate 1

Aggregate Repository Event StoreUpdate

append events

where rowId =

e8 e9

Event Store


Row ID Events

e1 e2 e3 e5 e6 e7e4 e8 e9

Aggregate RepositoryQuery

Aggregate Service

Get all aggregates for a customer


Aggregate Service Response

Aggregate 2Aggregate 1

Aggregate

DownloadedAggregate

License Aggregate

License Service

Acquire License

EndpointQuery

customerId: BayleytitleId: GlowS01E01

License Service

Query

customerId: BayleytitleId: GlowS01E01

Downloaded Service

isAllowed?

Query

customerId: BayleytitleId: GlowS01E01since: 6/27/2016

Downloaded Service

Downloaded Repository

getAggregates

Downloaded Service

Downloaded Repository

Response

DownloadedAggregate

customerId: BayleytitleId: GlowS01E01date: 2/15/2017

DownloadedAggregate


License Service

Downloaded Service

Response

isAllowed?

aggregates.size() < yearly limit

License Service

Downloaded Service

Response

isAllowed?

Trueaggregates.size() < yearly limit

(2) < (3)

License Service

Update

License Repository

Create Aggregate

Create License Command


License Aggregate

customerId:titleId:expires:released:

License Repository

Command HandlerCreate License

CommandcustomerId: BayleytitleId: GlowS01E01date: 6/27/2017

License Aggregate


License Repository

Command HandlerCreate License

CommandcustomerId: BayleytitleId: GlowS01E01date: 6/27/2017

License Aggregate

customerId:titleId:expires:released: License Created

Event


License Repository

Event Handler

License Aggregate


License Created Event


License Aggregate

customerId: BayleytitleId: GlowS01E01expires: 7/27/2017released: false

License Repository Event StoreSave

License Created Event


License Service

License Repository

Create Aggregate

CreatedAcquire License

EndpointResponse

License Service License Repository

Response

isAllowed?

aggregates.filter(GlowS01E01) .size()

< yearly limit

Get all licenses

Table

Partition Keys Clustering Columns Columns

Event Table

rowIdtext

Event Timetimestamp

Event Datablob

Aggregate Idtext

Event MapperString

● A fast and efficient object graph serialization framework for Java

○ https://github.com/EsotericSoftware/kryo

● Able to preconfigure the library to your data models for extra compaction

● Custom serializers

https://github.com/EsotericSoftware/kryo

https://github.com/EsotericSoftware/kryo

AggregateIdtext

Event Datablob

rowIdtext

Event Timetimestamp

House of Cards LicenseCreatedMatt:LicenseAggregate 0

Buddy Thunderstruck LicenseCreated1

The Hardy Boys LicenseCreatedJeff:LicenseAggregate 2

AggregateIdtext

Event Datablob

rowIdtext

Event Timetimestamp


House of Cards LicenseReleased3


The Hardy Boys LicenseCreatedJeff:LicenseAggregate 2

Snapshotting


House of Cards LicenseReleased3


Master of None LicenseCreated0

Master of None LicenseReleased3

Bojack LicenseCreated1

Lucha Underground LicenseCreated0

Lucha Underground LicenseReleased3

Grace & Frankie LicenseCreated1

House of Cards LicenseCreated0

Snapshot Table

rowIdtext

versionint

Snapshot Datablob

Versionint

Event Datablob

rowIdtext

1 binaryDataMatt:LicenseAggregate

AggregateIdtext

Event Datablob

rowIdtext

Event Timetimestamp

House of Cards LicenseCreatedMatt:LicenseAggregate:0 0

House of Cards LicenseReleasedMatt:LicenseAggregate:1 3

Jessica Jones LicenseCreated1

The Hardy Boys LicenseCreatedJeff:LicenseAggregate:0 2

Working With Event Sourcing

Flexibility in Practice

● Changes to the Data Model are trivial!○ Device deactivation

New Device Deactivation Requirement

Deactivate Device

EndpointDevice Service

Device Repository

Device Aggregate

Device Deactivated

EventDeactivate

Device Command

New Device Deactivation Requirement...

Deactivate Device

Endpoint

Device ServiceDeactivate Device

Can Deactivate?

DeviceRepository

Response

DeviceAggregate

CustomerId: BayleyDeviceId: iPhoneDeactivations: 1


Deactivate Device

Endpoint

Device ServiceDeactivate Device

Can Deactivate

License ServiceRelease Licenses

Device Repository

Command Handler

Deactivate Device CommandcustomerId: BayleydeviceId: iPhone

Device Deactivated Event

customerId: BayleydeviceId: iPhone

DeviceAggregate

CustomerId: BayleyDeviceId: iPhoneDeactivations: 1


Debugging

● Possibly the biggest win

● Current tooling is rudimentary

{ "Event" : "LicenseAcquiredEvent", "Event Time" : "2017-06-25 05:23:00 PM", "aggregateId" : "aggregateId1", "committed" : true, "eventTimeStamp" : 1498411380903, "eventOrderNumber" : 0}

Debugging...

Acquire License Event

customerId: BayleydeviceId: iPhonedate: 01/05/2017

Release License Event


Renew License Event


License Aggregate

customerId:Bayleyexpires: 01/07/2017released: True

Reliability: Fallbacks

Edge API Layer

License Accounting Service Client

License Accounting

Service CassandraFallback Response

How Did it Scale?

Predictive Scaling

Scalability with Cassandra SSD (I2) Nodes

Scalability with Cassandra SSD Nodes...

● Could handle much higher loads● Storage use ramping up very quickly

Storage Optimizations

● TTLs● HDD (D2) clusters:

○ More storage○ Higher latency (up to 1 second)

Storage Optimizations...

● Partitioned approach○ Snapshot and subsequent events saved to i2 (SSD) cluster○ Archive events to D2 (HDD) cluster

Snapshot DataVersion 3

Event DataVersion 3

Event DataVersion <= 2

i2

D2

Storage Optimizations...Partitioned approach using CQRS (Command Query Responsibility Segregation)

● Write segregation○ Event handlers that determine partitioned usage

● Query segregation○ Query SSD or HDD (or both)○ Uninterrupted event stream

Event Data

i2

D2

Snapshot + Events

Event Handler

Key Event Sourcing Take-Aways

● Flexible: Adapting to change can be simple



● Debugging: Debugging data / state transitions greatly simplified




● Reliable: Fallbacks provide service reliability




● Reliable: Fallbacks provide service reliability

● Scalable: Service scaled well, but good architectural solutions for data storage solutions should be considered.

Questions?

Phillipa AverySenior Software Engineer

Project Technical Lead and EngineerDownloads License Accounting

[email protected] / @PhillipaAvery

Robert RetaSenior Software Engineer

Event Sourcing System ArchitectDownloads License Accounting

[email protected] / @rreta04





presenters - qcon new york · presenters phillipa avery senior software engineer project technical...

Documents