Download - Building Applications with DynamoDB
![Page 1: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/1.jpg)
DynamoDB
Building Applicationswith
An Online Seminar - 16th May 2012
Dr Matt Wood, Amazon Web Services
![Page 2: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/2.jpg)
Thank you!
![Page 3: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/3.jpg)
Building Applications with DynamoDB
![Page 4: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/4.jpg)
Building Applications with DynamoDB
Getting started
![Page 5: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/5.jpg)
Building Applications with DynamoDB
Getting started
Data modeling
![Page 6: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/6.jpg)
Building Applications with DynamoDB
Getting started
Data modeling
Partitioning
![Page 7: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/7.jpg)
Building Applications with DynamoDB
Getting started
Data modeling
Partitioning
Analytics
![Page 8: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/8.jpg)
Getting started withDynamoDB
quick review
![Page 9: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/9.jpg)
DynamoDB is a managedNoSQL database service.
Store and retrieve any amount of data.Serve any level of request traffic.
![Page 10: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/10.jpg)
Without the operational burden.
![Page 11: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/11.jpg)
Consistent, predictableperformance.
Single digit millisecond latencies.Backed on solid-state drives.
![Page 12: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/12.jpg)
Flexible data model.
Key/attribute pairs. No schema required.
Easy to create. Easy to adjust.
![Page 13: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/13.jpg)
Seamless scalability.
No table size limits. Unlimited storage.No downtime.
![Page 14: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/14.jpg)
Durable.
Consistent, disk-only writes.
Replication across data centres andavailability zones.
![Page 15: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/15.jpg)
Without the operational burden.
![Page 16: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/16.jpg)
Without the operational burden.
FOCUS ON YOUR APP
![Page 17: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/17.jpg)
Two decisions + three clicks = ready for use
![Page 18: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/18.jpg)
Two decisions + three clicks = ready for use
Primary keys +
level of throughput
![Page 19: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/19.jpg)
Provisioned throughput.
Reserve IOPS for reads and writes.Scale up (or down) at any time.
![Page 20: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/20.jpg)
Pay per capacity unit.
Priced per hour of provisioned throughput.
![Page 21: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/21.jpg)
Write throughput.
$0.01 per hour for 10 write units
Units = size of item x writes/second
![Page 22: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/22.jpg)
Consistent writes.
Atomic increment/decrement.Optimistic concurrency control.aka: “conditional writes”.
![Page 23: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/23.jpg)
Transactions.
Item level transactions only.Puts, updates and deletes are ACID.
![Page 24: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/24.jpg)
Read throughput.
strongly consistent
eventually consistent
![Page 25: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/25.jpg)
Read throughput.
$0.01 per hour for 50 read units
Provisioned units = size of item x reads/second
strongly consistent
eventually consistent
![Page 26: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/26.jpg)
Read throughput.
$0.01 per hour for 100 read units
Provisioned units = size of item x reads/second
2
strongly consistent
eventually consistent
![Page 27: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/27.jpg)
Read throughput.
Mix and match at “read time”.
Same latency expectations.
strongly consistent
eventually consistent
![Page 28: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/28.jpg)
Two decisions + three clicks = ready for use
![Page 29: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/29.jpg)
![Page 30: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/30.jpg)
![Page 31: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/31.jpg)
![Page 32: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/32.jpg)
Two decisions + three clicks = ready for use
![Page 33: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/33.jpg)
Two decisions + one API call = ready for use
![Page 34: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/34.jpg)
$create_response = $dynamodb->create_table(array( 'TableName' => 'ProductCatalog', 'KeySchema' => array( 'HashKeyElement' => array( 'AttributeName' => 'Id', 'AttributeType' => AmazonDynamoDB::TYPE_NUMBER ) ), 'ProvisionedThroughput' => array( 'ReadCapacityUnits' => 10, 'WriteCapacityUnits' => 5 )));
![Page 35: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/35.jpg)
Two decisions + one API call = ready for use
![Page 36: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/36.jpg)
Two decisions + one API call = ready for development
![Page 37: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/37.jpg)
Two decisions + one API call = ready for production
![Page 38: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/38.jpg)
Two decisions + one API call = ready for scale
![Page 39: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/39.jpg)
![Page 40: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/40.jpg)
Authentication.
Session based to minimize latency.Uses Amazon Security Token Service.Handled by AWS SDKs.Integrates with IAM.
![Page 41: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/41.jpg)
Monitoring.
CloudWatch metrics: latency, consumed read and write throughput, errors and throttling.
![Page 42: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/42.jpg)
Libraries, mappers & mocks.
http://j.mp/dynamodb-libs
ColdFusion, Django, Erlang, Java, .Net,Node.js, Perl, PHP, Python, Ruby
![Page 43: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/43.jpg)
DynamoDB data models
![Page 44: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/44.jpg)
DynamoDB semantics.
Tables, items and attributes.
![Page 45: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/45.jpg)
Tables contain items.
Unlimited items per table.
![Page 46: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/46.jpg)
Items are a collection of attributes.
Each attribute has a key and a value.
An item can have any number ofattributes, up to 64k total.
![Page 47: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/47.jpg)
Two scalar data types.
String: Unicode, UTF8 binary encoding.Number: 38 digit precision.
Multi-value strings and numbers.
![Page 48: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/48.jpg)
id = 100 date = 2012-05-16-09-00-10 total = 25.00
id = 101 date = 2012-05-15-15-00-11 total = 35.00
id = 101 date = 2012-05-16-12-00-10 total = 100.00
id = 102 date = 2012-03-20-18-23-10 total = 20.00
id = 102 date = 2012-03-20-18-23-10 total = 120.00
![Page 49: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/49.jpg)
id = 100 date = 2012-05-16-09-00-10 total = 25.00
id = 101 date = 2012-05-15-15-00-11 total = 35.00
id = 101 date = 2012-05-16-12-00-10 total = 100.00
id = 102 date = 2012-03-20-18-23-10 total = 20.00
id = 102 date = 2012-03-20-18-23-10 total = 120.00
Table
![Page 50: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/50.jpg)
id = 100 date = 2012-05-16-09-00-10 total = 25.00
id = 101 date = 2012-05-15-15-00-11 total = 35.00
id = 101 date = 2012-05-16-12-00-10 total = 100.00
id = 102 date = 2012-03-20-18-23-10 total = 20.00
id = 102 date = 2012-03-20-18-23-10 total = 120.00
Item
![Page 51: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/51.jpg)
id = 100 date = 2012-05-16-09-00-10 total = 25.00
id = 101 date = 2012-05-15-15-00-11 total = 35.00
id = 101 date = 2012-05-16-12-00-10 total = 100.00
id = 102 date = 2012-03-20-18-23-10 total = 20.00
id = 102 date = 2012-03-20-18-23-10 total = 120.00
Attribute
![Page 52: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/52.jpg)
Where is the schema?
Tables do not require a formal schema.Items are an arbitrary sized hash.Just need to specify the primary key.
![Page 53: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/53.jpg)
Items are indexed by primary key.
Single hash keys and composite keys.
![Page 54: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/54.jpg)
id = 100 date = 2012-05-16-09-00-10 total = 25.00
id = 101 date = 2012-05-15-15-00-11 total = 35.00
id = 101 date = 2012-05-16-12-00-10 total = 100.00
id = 102 date = 2012-03-20-18-23-10 total = 20.00
id = 102 date = 2012-03-20-18-23-10 total = 120.00
Hash Key
![Page 55: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/55.jpg)
Range key for queries.
Querying items by composite key.
![Page 56: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/56.jpg)
id = 100 date = 2012-05-16-09-00-10 total = 25.00
id = 101 date = 2012-05-15-15-00-11 total = 35.00
id = 101 date = 2012-05-16-12-00-10 total = 100.00
id = 102 date = 2012-03-20-18-23-10 total = 20.00
id = 102 date = 2012-03-20-18-23-10 total = 120.00
Hash Key Range Key+
![Page 57: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/57.jpg)
Programming DynamoDB.
Small but perfectly formed.
Whole programming interface fits on one slide.
![Page 58: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/58.jpg)
CreateTable
UpdateTable
DeleteTable
DescribeTable
ListTables
PutItem
GetItem
UpdateItem
DeleteItem
BatchGetItem
BatchWriteItemQuery
Scan
![Page 59: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/59.jpg)
CreateTable
UpdateTable
DeleteTable
DescribeTable
ListTables
PutItem
GetItem
UpdateItem
DeleteItem
BatchGetItem
BatchWriteItemQuery
Scan
![Page 60: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/60.jpg)
CreateTable
UpdateTable
DeleteTable
DescribeTable
ListTables
PutItem
GetItem
UpdateItem
DeleteItem
BatchGetItem
BatchWriteItemQuery
Scan
![Page 61: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/61.jpg)
Conditional updates.
PutItem, UpdateItem, DeleteItem can take optional conditions for operation.
UpdateItem performs atomic increments.
![Page 62: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/62.jpg)
CreateTable
UpdateTable
DeleteTable
DescribeTable
ListTables
PutItem
GetItem
UpdateItem
DeleteItem
BatchGetItem
BatchWriteItemQuery
Scan
![Page 63: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/63.jpg)
One API call, multiple items.
BatchGet returns multiple items by primary key.
BatchWrite performs up to 25 put or delete operations.
Throughput is measured by IO, not API calls.
![Page 64: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/64.jpg)
CreateTable
UpdateTable
DeleteTable
DescribeTable
ListTables
PutItem
GetItem
UpdateItem
DeleteItem
BatchGetItem
BatchWriteItemQuery
Scan
![Page 65: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/65.jpg)
Query vs Scan
Query for composite key queries.Scan for full table scans, exports.
Both support pages and limits. Maximum response is 1Mb in size.
![Page 66: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/66.jpg)
Query patterns.
Retrieve all items by hash key.
Range key conditions: ==, <, >, >=, <=, begins with, between.
Counts. Top and bottom n values.Paged responses.
![Page 67: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/67.jpg)
Modeling patterns
![Page 68: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/68.jpg)
1. Mapping relationships with range keys.
No cross-table joins in DynamoDB.
Use composite keys to model relationships.
Patterns
![Page 69: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/69.jpg)
Data model example: online gaming.Storing scores and leader boards.
Players with high Scores.
Leader board for each game.
![Page 70: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/70.jpg)
Data model example: online gaming.Storing scores and leader boards.
Players with high Scores.
Leader board for each game.
user_id =mza
location =Cambridge
joined =2011-07-04
user_id =jeffbarr
location =Seattle
joined =2012-01-20
user_id =werner
location = Worldwide
joined = 2011-05-15
Players: hash key
![Page 71: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/71.jpg)
Data model example: online gaming.Storing scores and leader boards.
Players with high Scores.
Leader board for each game.
user_id =mza
location =Cambridge
joined =2011-07-04
user_id =jeffbarr
location =Seattle
joined =2012-01-20
user_id =werner
location = Worldwide
joined = 2011-05-15
Players: hash key
user_id =mza
game =angry-birds
score =11,000
user_id =mza
game =tetris
score =1,223,000
user_id =werner
location = bejewelled
score = 55,000
Scores: composite key
![Page 72: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/72.jpg)
Data model example: online gaming.Storing scores and leader boards.
Players with high Scores.
Leader board for each game.
user_id =mza
location =Cambridge
joined =2011-07-04
user_id =jeffbarr
location =Seattle
joined =2012-01-20
user_id =werner
location = Worldwide
joined = 2011-05-15
Players: hash key
user_id =mza
game =angry-birds
score =11,000
user_id =mza
game =tetris
score =1,223,000
user_id =werner
location = bejewelled
score = 55,000
Scores: composite key
game =angry-birds
score =11,000
user_id =mza
game =tetris
score =1,223,000
user_id =mza
game =tetris
score = 9,000,000
user_id = jeffbarr
Leader boards: composite key
![Page 73: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/73.jpg)
Data model example: online gaming.Storing scores and leader boards.
user_id =mza
location =Cambridge
joined =2011-07-04
user_id =jeffbarr
location =Seattle
joined =2012-01-20
user_id =werner
location = Worldwide
joined = 2011-05-15
Players: hash key
user_id =mza
game =angry-birds
score =11,000
user_id =mza
game =tetris
score =1,223,000
user_id =werner
location = bejewelled
score = 55,000
Scores: composite key
game =angry-birds
score =11,000
user_id =mza
game =tetris
score =1,223,000
user_id =mza
game =tetris
score = 9,000,000
user_id = jeffbarr
Leader boards: composite key
Scores by user(and by game)
![Page 74: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/74.jpg)
Data model example: online gaming.Storing scores and leader boards.
user_id =mza
location =Cambridge
joined =2011-07-04
user_id =jeffbarr
location =Seattle
joined =2012-01-20
user_id =werner
location = Worldwide
joined = 2011-05-15
Players: hash key
user_id =mza
game =angry-birds
score =11,000
user_id =mza
game =tetris
score =1,223,000
user_id =werner
location = bejewelled
score = 55,000
Scores: composite key
game =angry-birds
score =11,000
user_id =mza
game =tetris
score =1,223,000
user_id =mza
game =tetris
score = 9,000,000
user_id = jeffbarr
Leader boards: composite key
High scores by game
![Page 75: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/75.jpg)
2. Handling large items.
Unlimited attributes per item.Unlimited items per table.
Max 64k per item.
Patterns
![Page 76: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/76.jpg)
Data model example: large items.Storing more than 64k across items.
message_id =1
part =1
message = <first 64k>
message_id =1
part =2
message =<second 64k>
message_id =1
part =3
joined = <third 64k>
Large messages: composite keys
Split attributes across items.Query by message_id and part to retrieve.
![Page 77: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/77.jpg)
Store a pointer to objects in Amazon S3.
Large data stored in S3.Location stored in DynamoDB.
99.999999999% data durability in S3.
Patterns
![Page 78: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/78.jpg)
3. Managing secondary indices.
Not supported by DynamoDB.
Create your own.
Patterns
![Page 79: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/79.jpg)
Data model example: secondary indices.Storing more than 64k across items.
user_id =mza
first_name =Matt
last_name = Wood
user_id =mattfox
first_name =Matt
last_name =Fox
user_id =werner
first_name =Werner
last_name = Vogels
Users: hash key
![Page 80: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/80.jpg)
Data model example: secondary indices.Storing more than 64k across items.
user_id =mza
first_name =Matt
last_name = Wood
user_id =mattfox
first_name =Matt
last_name =Fox
user_id =werner
first_name =Werner
last_name = Vogels
Users: hash key
first_name =Matt
user_id =mza
first_name =Matt
user_id =mattfox
first_name = Werner
user_id =werner
First name index: composite keys
![Page 81: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/81.jpg)
Data model example: secondary indices.Storing more than 64k across items.
Users: hash key
first_name =Matt
user_id =mza
first_name =Matt
user_id =mattfox
first_name = Werner
user_id =werner
First name index: composite keys Second name index: composite keys
last_name =Wood
user_id =mza
last_name =Fox
user_id =mattfox
last_name = Vogels
user_id =werner
user_id =mza
first_name =Matt
last_name = Wood
user_id =mattfox
first_name =Matt
last_name =Fox
user_id =werner
first_name =Werner
last_name = Vogels
![Page 82: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/82.jpg)
last_name =Wood
user_id =mza
last_name =Fox
user_id =mattfox
last_name = Vogels
user_id =werner
user_id =mza
first_name =Matt
last_name = Wood
user_id =mattfox
first_name =Matt
last_name =Fox
user_id =werner
first_name =Werner
last_name = Vogels
Data model example: secondary indices.Storing more than 64k across items.
Users: hash key
first_name =Matt
user_id =mza
first_name =Matt
user_id =mattfox
first_name = Werner
user_id =werner
First name index: composite keys Second name index: composite keys
![Page 83: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/83.jpg)
last_name =Wood
user_id =mza
last_name =Fox
user_id =mattfox
last_name = Vogels
user_id =werner
user_id =mza
first_name =Matt
last_name = Wood
user_id =mattfox
first_name =Matt
last_name =Fox
user_id =werner
first_name =Werner
last_name = Vogels
Data model example: secondary indices.Storing more than 64k across items.
Users: hash key
first_name =Matt
user_id =mza
first_name =Matt
user_id =mattfox
first_name = Werner
user_id =werner
First name index: composite keys Second name index: composite keys
![Page 84: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/84.jpg)
4. Time series data.
Logging, click through, ad views, game play data, application usage.
Non-uniform access patterns.Newer data is ‘live’.Older data is read only.
Patterns
![Page 85: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/85.jpg)
Data model example: time series data.Rolling tables for hot and cold data.
event_id =1000
timestamp =2012-05-16-09-59-01
key = value
event_id =1001
timestamp =2012-05-16-09-59-02
key = value
event_id =1002
timestamp =2012-05-16-09-59-02
key = value
Events table: composite keys
![Page 86: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/86.jpg)
Data model example: time series data.Rolling tables for hot and cold data.
event_id =1000
timestamp =2012-05-16-09-59-01
key = value
event_id =1001
timestamp =2012-05-16-09-59-02
key = value
event_id =1002
timestamp =2012-05-16-09-59-02
key = value
Events table: composite keys
Events table for April: composite keys Events table for January: composite keys
event_id =400
timestamp =2012-04-01-00-00-01
event_id =401
timestamp =2012-04-01-00-00-02
event_id =402
timestamp =2012-04-01-00-00-03
event_id =100
timestamp =2012-01-01-00-00-01
event_id =101
timestamp =2012-01-01-00-00-02
event_id =102
timestamp =2012-01-01-00-00-03
![Page 87: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/87.jpg)
Hot and cold tables.
Jan April MayFeb MarDec
Patterns
![Page 88: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/88.jpg)
Hot and cold tables.
Jan April MayFeb Mar
higher throughput
Dec
Patterns
![Page 89: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/89.jpg)
Hot and cold tables.
Jan April MayFeb Mar
higher throughput
lower throughput
Dec
Patterns
![Page 90: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/90.jpg)
Hot and cold tables.
Jan April MayFeb Mar
data to S3, delete cold tables
Dec
Patterns
![Page 91: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/91.jpg)
Hot and cold tables.
Feb May JuneMar AprJan
Patterns
![Page 92: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/92.jpg)
Hot and cold tables.
Mar June JulyApr MayFeb
Patterns
![Page 93: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/93.jpg)
Hot and cold tables.
Apr July AugMay JuneMar
Patterns
![Page 94: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/94.jpg)
Hot and cold tables.
May Aug SeptJune JulyApr
Patterns
![Page 95: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/95.jpg)
Hot and cold tables.
June Sept OctJuly AugMay
Patterns
![Page 96: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/96.jpg)
Not out of mind.
DynamoDB and S3 data can be integrated for analytics.
Run queries across hot and cold data with Elastic MapReduce.
Patterns
![Page 97: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/97.jpg)
Partitioning best practices
![Page 98: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/98.jpg)
Uniform workloads.
DynamoDB divides table data into multiple partitions.
Data is distributed primarily byhash key.
Provisioned throughput is divided evenly across the partitions.
![Page 99: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/99.jpg)
Uniform workloads.
To achieve and maintain full provisioned throughput for a table, spread your workload evenly acrossthe hash keys.
![Page 100: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/100.jpg)
Non-uniform workloads.
Some requests might be throttled, even at high levels of provisioned throughput.
Some best practices...
![Page 101: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/101.jpg)
1. Distinct values for hash keys.
Patterns
Hash key elements should have a high number of distinct values.
![Page 102: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/102.jpg)
Data model example: hash key selection.Well distributed work loads
user_id =mza
first_name =Matt
last_name = Wood
user_id =jeffbarr
first_name =Jeff
last_name =Barr
user_id =werner
first_name =Werner
last_name = Vogels
user_id =mattfox
first_name =Matt
last_name = Fox
... ... ...
Users
![Page 103: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/103.jpg)
Data model example: hash key selection.Well distributed work loads
user_id =mza
first_name =Matt
last_name = Wood
user_id =jeffbarr
first_name =Jeff
last_name =Barr
user_id =werner
first_name =Werner
last_name = Vogels
user_id =mattfox
first_name =Matt
last_name = Fox
... ... ...
Users
Lots of users with unique user_id.Workload well distributed across user partitions.
![Page 104: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/104.jpg)
2. Avoid limited hash key values.
Patterns
Hash key elements should have a high number of distinct values.
![Page 105: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/105.jpg)
Data model example: small hash value range.Non-uniform workload.
status =200
date =2012-04-01-00-00-01
status =404
date = 2012-04-01-00-00-01
status404
date =2012-04-01-00-00-01
status =404
date =2012-04-01-00-00-01
Status responses
![Page 106: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/106.jpg)
Data model example: small hash value range.Non-uniform workload.
status =200
date =2012-04-01-00-00-01
status =404
date = 2012-04-01-00-00-01
status404
date =2012-04-01-00-00-01
status =404
date =2012-04-01-00-00-01
Status responses
Small number of status codes.Unevenly, non-uniform workload.
![Page 107: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/107.jpg)
3. Model for even distribution of access.
Patterns
Access by hash key value should be evenly distributed across the dataset.
![Page 108: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/108.jpg)
Data model example: uneven access pattern by key.Non-uniform access workload.
mobile_id =100
access_date =2012-04-01-00-00-01
mobile_id =100
access_date =2012-04-01-00-00-02
mobile_id =100
access_date =2012-04-01-00-00-03
mobile_id =100
access_date =2012-04-01-00-00-04
... ...
Devices
![Page 109: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/109.jpg)
mobile_id =100
access_date =2012-04-01-00-00-01
mobile_id =100
access_date =2012-04-01-00-00-02
mobile_id =100
access_date =2012-04-01-00-00-03
mobile_id =100
access_date =2012-04-01-00-00-04
... ...
Devices
Large number of devices.Small number which are much more popular than others.Workload unevenly distributed.
Data model example: uneven access pattern by key.Non-uniform access workload.
![Page 110: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/110.jpg)
mobile_id =100.1
access_date =2012-04-01-00-00-01
mobile_id =100.2
access_date =2012-04-01-00-00-02
mobile_id =100.3
access_date =2012-04-01-00-00-03
mobile_id =100.4
access_date =2012-04-01-00-00-04
... ...
Devices
Randomize access pattern.Workload randomised by hash key.
Data model example: randomize access pattern by key.Towards a uniform workload.
![Page 111: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/111.jpg)
Design for a uniform workload.
![Page 112: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/112.jpg)
Analytics with DynamoDB
![Page 113: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/113.jpg)
Seamless scale.
Scalable methods for data processing.Scalable methods for backup/restore.
![Page 114: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/114.jpg)
Amazon Elastic MapReduce.
http://aws.amazon.com/emr
Managed Hadoop service for data-intensive workflows.
![Page 115: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/115.jpg)
Hadoop under the hood.
Take advantage of the Hadoop ecosystem: streaming interfaces,Hive, Pig, Mahout.
![Page 116: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/116.jpg)
Distributed data processing.
API driven. Analytics at any scale.
![Page 117: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/117.jpg)
Query flexibility with Hive.
create external table items_db (id string, votes bigint, views bigint) stored by 'org.apache.hadoop.hive.dynamodb.DynamoDBStorageHandler' tblproperties ("dynamodb.table.name" = "items", "dynamodb.column.mapping" = "id:id,votes:votes,views:views");
![Page 118: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/118.jpg)
Query flexibility with Hive.
select id, likes, views from items_db order by views desc;
![Page 119: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/119.jpg)
Data export/import.
Use EMR for backup and restore to Amazon S3.
![Page 120: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/120.jpg)
Data export/import.CREATE EXTERNAL TABLE orders_s3_new_export ( order_id string, customer_id string, order_date int, total double )PARTITIONED BY (year string, month string)ROW FORMAT DELIMITED FIELDS TERMINATED BY ','LOCATION 's3://export_bucket';
INSERT OVERWRITE TABLE orders_s3_new_exportPARTITION (year='2012', month='01')SELECT * from orders_ddb_2012_01;
![Page 121: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/121.jpg)
Integrate live and archive data
Run queries across external Hive tableson S3 and DynamoDB.
Live & archive. Metadata & big objects.
![Page 122: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/122.jpg)
In summary...
DynamoDB
Predictable performance
Provisioned throughput
Libraries & mappers
![Page 123: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/123.jpg)
In summary...
DynamoDB
Data modeling
Predictable performance
Provisioned throughput
Libraries & mappers
Tables & items
Read & write patterns
Time series data
![Page 124: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/124.jpg)
In summary...
DynamoDB
Data modeling
PartitioningPredictable performance
Provisioned throughput
Libraries & mappers
Tables & items
Read & write patterns
Time series data
Automatic partitioning
Hot and cold data
Size/throughput ratio
![Page 125: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/125.jpg)
In summary...
DynamoDB
Data modeling
Partitioning
Analytics
Predictable performance
Provisioned throughput
Libraries & mappers
Tables & items
Read & write patterns
Time series data
Automatic partitioning
Hot and cold data
Size/throughput ratio
Elastic MapReduce
Hive queries
Backup & restore
![Page 126: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/126.jpg)
DynamoDB free tier
5 writes, 10 consistent reads per second100Mb of storage
![Page 127: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/127.jpg)
aws.amazon.com/dynamodb
aws.amazon.com/documentation/dynamodb
best practice + sample code
![Page 128: Building Applications with DynamoDB](https://reader034.vdocument.in/reader034/viewer/2022042521/54c6a8f84a7959b14b8b4569/html5/thumbnails/128.jpg)
Thank you!