![Page 1: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/1.jpg)
Under the Covers of DynamoDB
Philip Fitzsimons
Manager, Solution Architecture
![Page 2: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/2.jpg)
1. Getting started
3. Data modeling
4. Partitioning
5. Reporting & Analytics
Overview
2. Customer story: Royal Opera House
![Page 3: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/3.jpg)
Getting started
![Page 4: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/4.jpg)
DynamoDB is a managed
NoSQL database service.
Store and retrieve any amount of data.
Serve any level of request traffic.
![Page 5: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/5.jpg)
Without the operational burden.
![Page 6: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/6.jpg)
Consistent, predictable performance.
Single digit millisecond latency.
Backed on solid-state drives.
![Page 7: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/7.jpg)
Flexible data model.
Key/attribute pairs. No schema required.
Easy to create. Easy to adjust.
![Page 8: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/8.jpg)
Seamless scalability.
No table size limits. Unlimited storage.
No downtime.
![Page 9: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/9.jpg)
Durable.
Consistent, disk only writes.
Replication across data centers and availability zones.
![Page 10: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/10.jpg)
Without the operational burden.
![Page 11: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/11.jpg)
Focus on your app.
![Page 12: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/12.jpg)
Two decisions + three clicks
= ready for use
![Page 13: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/13.jpg)
Two decisions + three clicks
= ready for use
Primary keys
Level of throughput
![Page 14: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/14.jpg)
Two decisions + three clicks
= ready for use
Primary keys
Level of throughput
![Page 15: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/15.jpg)
Provisioned throughput.
Reserve IOPS for reads and writes.
Scale up for down at any time.
![Page 16: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/16.jpg)
Pay per capacity unit.
Priced per hour of provisioned throughput.
![Page 17: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/17.jpg)
Write throughput.
Size of item x writes per second
$0.0065 for 10 write units
![Page 18: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/18.jpg)
Consistent writes.
Atomic increment and decrement.
Optimistic concurrency control: conditional writes.
![Page 19: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/19.jpg)
Transactions.
Item level transactions only.
Puts, updates and deletes are ACID.
![Page 20: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/20.jpg)
Read throughput.
Strong or eventual consistency
![Page 21: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/21.jpg)
Read throughput.
Strong or eventual consistency
Provisioned units = size of item x reads per second
$0.0065 per hour for 50 units
![Page 22: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/22.jpg)
Read throughput.
Strong or eventual consistency
Provisioned units = size of item x reads per second
$0.0065 per hour for 100 units
2
![Page 23: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/23.jpg)
Read throughput.
Strong or eventual consistency
Same latency expectations.
Mix and match at ‘read time’.
![Page 24: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/24.jpg)
Customer story
Royal Opera House
![Page 25: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/25.jpg)
Royal Opera House
Rob Greig
CTO @rob_greig
![Page 26: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/26.jpg)
Strategy
Innovation using open data and collaborative working
Web 3.0?
![Page 27: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/27.jpg)
>600 pages
Inconsistent structure
…and this doesn’t include microsites
![Page 28: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/28.jpg)
Manual interlinking
Poor discoverability
![Page 29: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/29.jpg)
Coherent and consistent linking
throughout the complete
production lifecycle
![Page 30: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/30.jpg)
iPad
![Page 31: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/31.jpg)
Open Data
![Page 32: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/32.jpg)
![Page 33: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/33.jpg)
![Page 34: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/34.jpg)
Customer Service
![Page 35: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/35.jpg)
![Page 36: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/36.jpg)
DynamoDB
![Page 37: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/37.jpg)
![Page 38: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/38.jpg)
![Page 39: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/39.jpg)
![Page 40: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/40.jpg)
![Page 41: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/41.jpg)
Provisioned throughput is
managed by DynamoDB.
![Page 42: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/42.jpg)
Data is partitioned and
managed by DynamoDB.
![Page 43: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/43.jpg)
Reserved capacity.
Up to 53% for 1 year reservation.
Up to 76% for 3 year reservation.
![Page 44: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/44.jpg)
Authentication.
Session based to minimize latency.
Uses the Amazon Security Token Service.
Handled by AWS SDKs.
Integrates with IAM.
![Page 45: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/45.jpg)
Monitoring.
CloudWatch metrics:
latency, consumed read and write throughput,
errors and throttling.
![Page 46: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/46.jpg)
Libraries, mappers and mocks.
ColdFusion, Django, Erlang, Java, .Net,
Node.js, Perl, PHP, Python, Ruby
http://j.mp/dynamodb-libs
![Page 47: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/47.jpg)
Data Modeling
![Page 48: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/48.jpg)
id = 100 date = 2012-05-16-
09-00-10 total = 25.00
id = 101 date = 2012-05-15-
15-00-11 total = 35.00
id = 101 date = 2012-05-16-
12-00-10 total = 100.00
id = 102 date = 2012-03-20-
18-23-10 total = 20.00
id = 102 date = 2012-03-20-
18-23-10 total = 120.00
![Page 49: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/49.jpg)
id = 100 date = 2012-05-16-
09-00-10 total = 25.00
id = 101 date = 2012-05-15-
15-00-11 total = 35.00
id = 101 date = 2012-05-16-
12-00-10 total = 100.00
id = 102 date = 2012-03-20-
18-23-10 total = 20.00
id = 102 date = 2012-03-20-
18-23-10 total = 120.00
Table
![Page 50: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/50.jpg)
id = 100 date = 2012-05-16-
09-00-10 total = 25.00
id = 101 date = 2012-05-15-
15-00-11 total = 35.00
id = 101 date = 2012-05-16-
12-00-10 total = 100.00
id = 102 date = 2012-03-20-
18-23-10 total = 20.00
id = 102 date = 2012-03-20-
18-23-10 total = 120.00
Item
![Page 51: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/51.jpg)
id = 100 date = 2012-05-16-
09-00-10 total = 25.00
id = 101 date = 2012-05-15-
15-00-11 total = 35.00
id = 101 date = 2012-05-16-
12-00-10 total = 100.00
id = 102 date = 2012-03-20-
18-23-10 total = 20.00
id = 102 date = 2012-03-20-
18-23-10 total = 120.00
Attribute
![Page 52: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/52.jpg)
Where is the schema?
Tables do not require a formal schema.
Items are an arbitrarily sized hash.
![Page 53: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/53.jpg)
Indexing.
Items are indexed by primary and secondary keys.
Primary keys can be composite.
Secondary keys index on other attributes.
![Page 54: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/54.jpg)
ID Date Total
id = 100 date = 2012-05-16-09-00-10 total = 25.00
id = 101 date = 2012-05-15-15-00-11 total = 35.00
id = 101 date = 2012-05-16-12-00-10 total = 100.00
id = 102 date = 2012-03-20-18-23-10 total = 20.00
id = 102 date = 2012-03-20-18-23-10 total = 120.00
![Page 55: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/55.jpg)
ID Date Total
id = 100 date = 2012-05-16-09-00-10 total = 25.00
id = 101 date = 2012-05-15-15-00-11 total = 35.00
id = 101 date = 2012-05-16-12-00-10 total = 100.00
id = 102 date = 2012-03-20-18-23-10 total = 20.00
id = 102 date = 2012-03-20-18-23-10 total = 120.00
Hash key
![Page 56: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/56.jpg)
ID Date Total
id = 100 date = 2012-05-16-09-00-10 total = 25.00
id = 101 date = 2012-05-15-15-00-11 total = 35.00
id = 101 date = 2012-05-16-12-00-10 total = 100.00
id = 102 date = 2012-03-20-18-23-10 total = 20.00
id = 102 date = 2012-03-20-18-23-10 total = 120.00
Hash key Range key
Composite primary key
![Page 57: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/57.jpg)
ID Date Total
id = 100 date = 2012-05-16-09-00-10 total = 25.00
id = 101 date = 2012-05-15-15-00-11 total = 35.00
id = 101 date = 2012-05-16-12-00-10 total = 100.00
id = 102 date = 2012-03-20-18-23-10 total = 20.00
id = 102 date = 2012-03-20-18-23-10 total = 120.00
Hash key Range key Secondary range key
![Page 58: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/58.jpg)
Programming DynamoDB.
Small but perfectly formed API.
![Page 59: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/59.jpg)
CreateTable
UpdateTable
DeleteTable
DescribeTable
ListTables
Query
Scan
PutItem
GetItem
UpdateItem
DeleteItem
BatchGetItem
BatchWriteItem
![Page 60: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/60.jpg)
CreateTable
UpdateTable
DeleteTable
DescribeTable
ListTables
Query
Scan
PutItem
GetItem
UpdateItem
DeleteItem
BatchGetItem
BatchWriteItem
![Page 61: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/61.jpg)
CreateTable
UpdateTable
DeleteTable
DescribeTable
ListTables
Query
Scan
PutItem
GetItem
UpdateItem
DeleteItem
BatchGetItem
BatchWriteItem
![Page 62: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/62.jpg)
Conditional updates.
PutItem, UpdateItem, DeleteItem can take
optional conditions for operation.
UpdateItem performs atomic increments.
![Page 63: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/63.jpg)
One API call, multiple items
BatchGet returns multiple items by key.
Throughput is measured by IO, not API calls.
BatchWrite performs up to 25 put or delete operations.
![Page 64: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/64.jpg)
CreateTable
UpdateTable
DeleteTable
DescribeTable
ListTables
Query
Scan
PutItem
GetItem
UpdateItem
DeleteItem
BatchGetItem
BatchWriteItem
![Page 65: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/65.jpg)
Query vs Scan
Query for Composite Key queries.
Scan for full table scans, exports.
Both support pages and limits.
Maximum response is 1Mb in size.
![Page 66: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/66.jpg)
Query patterns
Retrieve all items by hash key.
Range key conditions:
==, <, >, >=, <=, begins with, between.
Counts. Top and bottom n values.
Paged responses.
![Page 67: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/67.jpg)
Mapping relationships.
EXAMPLE 1:
![Page 68: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/68.jpg)
Players
user_id =
mza
location =
Cambridge
joined =
2011-07-04
user_id =
jeffbarr
location =
Seattle
joined =
2012-01-20
user_id =
werner
location =
Worldwide
joined =
2011-05-15
![Page 69: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/69.jpg)
Players
user_id =
mza
location =
Cambridge
joined =
2011-07-04
user_id =
jeffbarr
location =
Seattle
joined =
2012-01-20
user_id =
werner
location =
Worldwide
joined =
2011-05-15
Scores user_id =
mza
game =
angry-birds
score =
11,000
user_id =
mza
game =
tetris
score =
1,223,000
user_id =
werner
location =
bejewelled
score =
55,000
![Page 70: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/70.jpg)
Players
user_id =
mza
location =
Cambridge
joined =
2011-07-04
user_id =
jeffbarr
location =
Seattle
joined =
2012-01-20
user_id =
werner
location =
Worldwide
joined =
2011-05-15
Scores Leader boards
user_id =
mza
game =
angry-birds
score =
11,000
user_id =
mza
game =
tetris
score =
1,223,000
user_id =
werner
location =
bejewelled
score =
55,000
game =
angry-birds
score =
11,000
user_id =
mza
game =
tetris
score =
1,223,000
user_id =
mza
game =
tetris
score =
9,000,000
user_id =
jeffbarr
![Page 71: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/71.jpg)
Players
user_id =
mza
location =
Cambridge
joined =
2011-07-04
user_id =
jeffbarr
location =
Seattle
joined =
2012-01-20
user_id =
werner
location =
Worldwide
joined =
2011-05-15
user_id =
mza
game =
angry-birds
score =
11,000
user_id =
mza
game =
tetris
score =
1,223,000
user_id =
werner
location =
bejewelled
score =
55,000
Scores game =
angry-birds
score =
11,000
user_id =
mza
game =
tetris
score =
1,223,000
user_id =
mza
game =
tetris
score =
9,000,000
user_id =
jeffbarr
Leader boards
Query for scores
by user
![Page 72: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/72.jpg)
Players
user_id =
mza
location =
Cambridge
joined =
2011-07-04
user_id =
jeffbarr
location =
Seattle
joined =
2012-01-20
user_id =
werner
location =
Worldwide
joined =
2011-05-15
Scores Leader boards
user_id =
mza
game =
angry-birds
score =
11,000
user_id =
mza
game =
tetris
score =
1,223,000
user_id =
werner
location =
bejewelled
score =
55,000
game =
angry-birds
score =
11,000
user_id =
mza
game =
tetris
score =
1,223,000
user_id =
mza
game =
tetris
score =
9,000,000
user_id =
jeffbarr
High scores by game
![Page 73: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/73.jpg)
Storing large items.
EXAMPLE 2:
![Page 74: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/74.jpg)
Unlimited storage.
Unlimited attributes per item.
Unlimited items per table.
Maximum of 64k per item.
![Page 75: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/75.jpg)
message_id = 1 part = 1 message =
<first 64k>
message_id = 1 part = 2 message =
<second 64k>
message_id = 1 part = 3 joined =
<third 64k>
Split across items.
![Page 76: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/76.jpg)
message_id = 1 message =
http://s3.amazonaws.com...
message_id = 2 message =
http://s3.amazonaws.com...
message_id = 3 message =
http://s3.amazonaws.com...
Store a pointer to S3.
![Page 77: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/77.jpg)
Time series data
EXAMPLE 3:
![Page 78: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/78.jpg)
event_id =
1000
timestamp =
2013-04-16-09-59-01
key =
value
event_id =
1001
timestamp =
2013-04-16-09-59-02
key =
value
event_id =
1002
timestamp =
2013-04-16-09-59-02
key =
value
Hot and cold tables. April
March
event_id =
1000
timestamp =
2013-03-01-09-59-01
key =
value
event_id =
1001
timestamp =
2013-03-01-09-59-02
key =
value
event_id =
1002
timestamp =
2013-03-01-09-59-02
key =
value
![Page 79: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/79.jpg)
April March February January December
![Page 80: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/80.jpg)
Archive data.
Move old data to S3: lower cost.
Still available for analytics.
Run queries across hot and cold data
with Elastic MapReduce.
![Page 81: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/81.jpg)
Partitioning
![Page 82: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/82.jpg)
Uniform workload.
Data stored across multiple partitions.
Data is primarily distributed by primary key.
Provisioned throughput is divided evenly across partitions.
![Page 83: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/83.jpg)
To achieve and maintain full
provisioned throughput, spread
workload evenly across hash keys.
![Page 84: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/84.jpg)
Non-Uniform workload.
Might be throttled, even at high levels of throughput.
![Page 85: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/85.jpg)
Distinct values for hash keys.
BEST PRACTICE 1:
Hash key elements should have a
high number of distinct values.
![Page 86: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/86.jpg)
user_id =
mza
first_name =
Matt
last_name =
Wood
user_id =
jeffbarr
first_name =
Jeff
last_name =
Barr
user_id =
werner
first_name =
Werner
last_name =
Vogels
user_id =
simone
first_name =
Simone
last_name =
Brunozzi
... ... ...
Lots of users with unique user_id.
Workload well distributed across hash key.
![Page 87: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/87.jpg)
Avoid limited hash key values.
BEST PRACTICE 2:
Hash key elements should have a
high number of distinct values.
![Page 88: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/88.jpg)
status =
200
date =
2012-04-01-00-00-01
status =
404
date =
2012-04-01-00-00-01
status
404
date =
2012-04-01-00-00-01
status =
404
date =
2012-04-01-00-00-01
Small number of status codes.
Unevenly, non-uniform workload.
![Page 89: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/89.jpg)
Model for even distribution.
BEST PRACTICE 3:
Access by hash key value should be evenly
distributed across the dataset.
![Page 90: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/90.jpg)
mobile_id =
100
access_date =
2012-04-01-00-00-01
mobile_id =
100
access_date =
2012-04-01-00-00-02
mobile_id =
100
access_date =
2012-04-01-00-00-03
mobile_id =
100
access_date =
2012-04-01-00-00-04
... ...
Large number of devices.
Small number which are much more popular than others.
Workload unevenly distributed.
![Page 91: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/91.jpg)
mobile_id =
100.1
access_date =
2012-04-01-00-00-01
mobile_id =
100.2
access_date =
2012-04-01-00-00-02
mobile_id =
100.3
access_date =
2012-04-01-00-00-03
mobile_id =
100.4
access_date =
2012-04-01-00-00-04
... ...
Sample access pattern.
Workload randomized by hash key.
![Page 92: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/92.jpg)
Reporting & Analytics
![Page 93: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/93.jpg)
Seamless scale.
Scalable methods for data processing.
Scalable methods for backup/restore.
![Page 94: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/94.jpg)
Amazon Elastic MapReduce.
Managed Hadoop service for
data-intensive workflows.
aws.amazon.com/emr
![Page 95: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/95.jpg)
create external table items_db
(id string, votes bigint, views bigint) stored by
'org.apache.hadoop.hive.dynamodb.DynamoDBStorageHandler'
tblproperties
("dynamodb.table.name" = "items",
"dynamodb.column.mapping" =
"id:id,votes:votes,views:views");
![Page 96: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/96.jpg)
select id, likes, views
from items_db
order by views desc;
![Page 97: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/97.jpg)
Summary
1. Getting started
3. Data modeling
4. Partitioning
5. Reporting & Analytics
2. Customer story: Royal Opera House
![Page 98: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/98.jpg)
Free tier.
![Page 99: Data & Analytics - Session 3 - Under the Covers with Amazon DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022051613/54c6a94f4a7959b14b8b4578/html5/thumbnails/99.jpg)
aws.amazon.com/dynamodb