Download - SQL or NoSQL - TrueNorthPHP
S Q LORNoSQL
That’sT H E Ques.on
@majidfnMajiD FatemianUbisoft Montreal
BIGDATA
NoSQL
1 2 3
No SQL Cluster Schemaless
Fitness Evolved
Fitness Evolved
2.2M 10K 160K(Max)40K (Avg)
Users Concurrent Daily Activities
STATS
2011
2012
2013
2011
2012
2013
Architecture2011
SQL / JOINACID
TransactionPartitioningReplication
ToolsOPs
Data Modeling
+
2011
2012
2013
Profile Info.
SpecificGlobal
Aggregated stats Activities
DATA
Community Events
Details
Leaderboards
SQL / JOINACID
TransactionPartitioningReplication
ToolsOPs
Data Modeling
VolumeNo DetailsVariant SchemaPartitioningScale outSharding
+ -
SQL / JOINACID
TransactionPartitioningReplication
ToolsOPs
Data Modeling
VolumeNo DetailsVariant SchemaPartitioningScale outSharding
+ -{ player_id: 1234, game_id: 4001, platform: xbox360, calories: 40, duration: 75, difficulty_level: medium, jumping_ropes: 178, score: 450, endurance:3, category: 4}
{ player_id: 1234, game_id: 1002, platform: wiiu, calories: 75, duration: 120} *
***
xbox v.1 xbox v.2Variant Schema
SQL / JOINACID
TransactionPartitioningReplication
ToolsOPs
Data Modeling
VolumeNo DetailsVariant SchemaPartitioningScale outSharding
+ -Scale out
SQL / JOINACID
TransactionPartitioningReplication
ToolsOPs
Data Modeling
VolumeNo DetailsVariant SchemaPartitioningScale outSharding
+ -Sharding
Data CData B
Data A
Data Access Layer
SQL / JOINACID
TransactionPartitioningReplication
ToolsOPs
Data Modeling
VolumeNo DetailsVariant SchemaPartitioningScale outSharding
+ -
NoSQL
...
mongoDB
1 2 3
OPs Support Community
Schema-LessAggregated DataObject MappingLarge Scale Data
ShardingMap/Reduce
Eventual ConsistencyMemory Storage
Journaling
+
Aggregated DataObject MappingLarge Scale Data
ShardingMap/Reduce
Eventual ConsistencyMemory Storage
Journaling
Schema-Less
{ player_id: 1234, game_id: 4001, platform: xbox360, calories: 40, duration: 75, difficulty_level: medium, jumping_ropes: 178, score: 450, endurance:3, category: 4}
{ player_id: 1234, game_id: 1002, platform: wiiu, calories: 75, duration: 120} *
***
xbox v.1 WiiU
+
Schema-Less
Large Scale DataSharding
Map/ReduceEventual Consistency
Memory StorageJournaling
Aggregated DataObject Mapping
{ player: { name: example, gender: M, country:CA } game_id: 4001, date: “2013-11-08”, calories: 40, duration: 75,}
+
Schema-LessAggregated DataObject Mapping
Eventual ConsistencyMemory Storage
Journaling
+
Large Scale DataSharding
Map/Reduce
Data AData B
Data C
Schema-LessAggregated DataObject MappingLarge Scale Data
ShardingMap/Reduce
Eventual ConsistencyMemory Storage
Journaling
+
Denormalized dataDisk SpaceExpertiseComplex QueryingResource UsageDB-Level Locking
-
Schema-LessAggregated DataObject MappingLarge Scale Data
ShardingMap/Reduce
Eventual ConsistencyMemory Storage
Journaling
+
BENCHMARK
Random ProfileSearch
(1M)
Full HistoryRetrieval
(6M)
Insert(6M)
AggregationMap/Reduce
(6M)
0
0.005
0.01
0.015
0.02
MySQLMongoDB
Relational Data ModelEase of Use
Schema LessPerformanceSharding
GO!Hybrid
Master
Slave
Config Server
Shard 1Replicaset
Shard 2Replicaset
Concerns
Inconsistency!
2
3
4
Data Duplication
Consistency vs. Latency
Complexity
Challenges
Availability
! 2
Time
3
Expertese
Data Access Layer
OR
FailureBackup
Plans
DATAMigration
1 One Shot Migration
2 Profile Versioning
4 hrs !!Live!
Read
Write
TEST!
2Functional
3Integration
1Unit
FinalResults
Scalability
&...
B.I.User
Details
Response Time
0.02 0.0002” ”
Architecture2013
REST
> 1 year uptime
1
2
Based on Requirements
Benchmark!
3 Choose Wisely!
References
NoSQLDistilled
MongoDBThe definitive guide
MySQLHigh Performance
we needYOU!
http://montreal.ubisoft.com/en/video-games-jobs-my-career
@majidfn
Q?Got