dynamodb at hasoffers
DESCRIPTION
How HasOffers uses DynamoDBTRANSCRIPT
DynamoDB at HasOffers
• Attribution for web & mobile marketing• Tons of data• Ad-hoc analysis• Near real-time
18 months in production
207 billion items
87 TB of data
DynamoDB at HasOffers
Tracking sessions
Deduplication
Attribution
Use Cases
Tracking Sessions
Session Data
Use Case: Tracking Sessions
671c6a9a-bed0-4cec-a465-613cca8c9832
• Impression• Click
Use Case: Tracking Sessions
• Ideal for NoSQL
• KV lookups
• Scalability
• Availability
Deduplication
Use Case: Event Deduplication671c6a9a-bed0-4cec-a465-613cca8c9832
ebe0efef-aa71-471c-b3f8-7b66b458bd79
ebe0efef-aa71-471c-b3f8-7b66b458bd79
d149e52c-d2b3-49bc-b6af-f304b5fab57f
DEDUPE!
(Idempotency)
Use Case: Event Deduplication
Hash uuidRange <none>
Write capacity = X, Read capacity = 1
Conditional puts
Use Case: Event Deduplication
DEDUPE!
def is_duplicate( dynamo_tbl, uuid ):
row_data = {‘host’: socket.gethostname(), ‘created’: int(time.time())}
row = dynamo_tbl.new_item( hash_key=uuid, attrs=row_data )
try: row.put( expected_value={‘host’: False} )
except DynamoDBConditionalCheckFailedError: return True
# not a duplicate return False
Use Case: Event Deduplication
DEDUPE!
• Real-time dedupe• Conditional put• Writes only; no read• Extra debug info
{uuid: d149e52c-d2b3-49bc-b6af-f304b5fab57f, host: server03.hasoffers.com, created: 1398884990}
Why NoSQL SaaS?
• Upsize and downsize trivial in DynamoDB
• Infrequent on own cluster = limits operational experience
Cluster Resizing
Cluster Software Updates
0.7 1.0
• Production depends on NoSQL solution
being available.
• Software version updates somewhere
between nail-biting and terrifying
Dynamo Operations
Per table R/W capacity
Capacity units
Exceed = throttling
DynamoDB Capacity
DynamoDB Throttling
CloudWatch = 5 minutes
DynamoDB = 1 second
DynamoDB Throttling
• Double capacity on table
• Root cause from log analysis– capacity usage
– key hot-spotting
• Fix root cause
• Reduce capacity on table
Closing Thoughts
• Compelling offering
• Simple operations
• Avoid hammer syndrome
• Very happy
Closing Thoughts
