Download - Scaling Online Social Networks (OSNs)
![Page 1: Scaling Online Social Networks (OSNs)](https://reader034.vdocument.in/reader034/viewer/2022042614/556aead3d8b42a86218b4d07/html5/thumbnails/1.jpg)
Presented by: Maria Stylianou Coworker: Anis Uddin
Supervisor: Šarūnas Girdzijauskas
KTH - Royal Institute of TechnologyImplementation of Distributed Systems
December 6th, 2012
Scaling Online Social Networks (OSNs)
![Page 2: Scaling Online Social Networks (OSNs)](https://reader034.vdocument.in/reader034/viewer/2022042614/556aead3d8b42a86218b4d07/html5/thumbnails/2.jpg)
2
Outline
● Motivation● Current Algorithms
– SPAR
– JA-BE-JA
● Contributions– Challenges
– Solution
● Evaluation & Conclusions
![Page 3: Scaling Online Social Networks (OSNs)](https://reader034.vdocument.in/reader034/viewer/2022042614/556aead3d8b42a86218b4d07/html5/thumbnails/3.jpg)
3
Outline
● Motivation● Current Algorithms
– SPAR
– JA-BE-JA
● Contributions– Challenges
– Solution
● Evaluation & Conclusions
![Page 4: Scaling Online Social Networks (OSNs)](https://reader034.vdocument.in/reader034/viewer/2022042614/556aead3d8b42a86218b4d07/html5/thumbnails/4.jpg)
4
“Pandora's box”Online Social Networks
Motivation-Algorithms-Contribution-Evaluation
Source: http://technorati.com/social-media/article/social-networks-theyre-what-every-local/
![Page 5: Scaling Online Social Networks (OSNs)](https://reader034.vdocument.in/reader034/viewer/2022042614/556aead3d8b42a86218b4d07/html5/thumbnails/5.jpg)
5
Easy to maintain...Online Social Networks
Motivation-Algorithms-Contribution-Evaluation
Source: http://mastersofmedia.hum.uva.nl/2009/09/14/a-review-of-taken-out-of-context/
![Page 6: Scaling Online Social Networks (OSNs)](https://reader034.vdocument.in/reader034/viewer/2022042614/556aead3d8b42a86218b4d07/html5/thumbnails/6.jpg)
6
...or not!Online Social Networks
Motivation-Algorithms-Contribution-Evaluation
Source: http://mastersofmedia.hum.uva.nl/2009/09/14/a-review-of-taken-out-of-context/
![Page 7: Scaling Online Social Networks (OSNs)](https://reader034.vdocument.in/reader034/viewer/2022042614/556aead3d8b42a86218b4d07/html5/thumbnails/7.jpg)
7
Scaling Approaches
Vertical Scaling● Full Replication
● Data Locality
● But:
– Expensive
– Saturation
Motivation-Algorithms-Contribution-Evaluation
Horizontal Scaling● Adding servers
● Clean & Disjoint Partitions
● But:
– Not applicable in OSNs
![Page 8: Scaling Online Social Networks (OSNs)](https://reader034.vdocument.in/reader034/viewer/2022042614/556aead3d8b42a86218b4d07/html5/thumbnails/8.jpg)
8
Scaling Approaches
Vertical Scaling● Full Replication
● Data Locality
● But:
– Expensive
– Saturation
Motivation-Algorithms-Contribution-Evaluation
Inefficient
Horizontal Scaling● Adding servers
● Clean & Disjoint Partitions
● But:
– Not applicable in OSNs
![Page 9: Scaling Online Social Networks (OSNs)](https://reader034.vdocument.in/reader034/viewer/2022042614/556aead3d8b42a86218b4d07/html5/thumbnails/9.jpg)
9
Existing 'Solutions' for OSNs
Relational Databases
Motivation-Algorithms-Contribution-Evaluation
Key-Value Stores
![Page 10: Scaling Online Social Networks (OSNs)](https://reader034.vdocument.in/reader034/viewer/2022042614/556aead3d8b42a86218b4d07/html5/thumbnails/10.jpg)
10
Existing 'Solutions' for OSNs
Relational Databases
Motivation-Algorithms-Contribution-Evaluation
Inefficient
Key-Value Stores
![Page 11: Scaling Online Social Networks (OSNs)](https://reader034.vdocument.in/reader034/viewer/2022042614/556aead3d8b42a86218b4d07/html5/thumbnails/11.jpg)
11
Outline
● Motivation● Current Algorithms
– SPAR
– JA-BE-JA
● Contributions– Challenges
– Solutions
● Evaluation & Conclusions
![Page 12: Scaling Online Social Networks (OSNs)](https://reader034.vdocument.in/reader034/viewer/2022042614/556aead3d8b42a86218b4d07/html5/thumbnails/12.jpg)
12
SPAR
Social Partitioning & Replication middle-ware● Transparent OSN scalability avoids ● Data Locality performance● Load Balancing bottlenecks
● Fault Tolerance● Stability● Replication Overhead Minimization
Motivation-Algorithms-Contribution-Evaluation
![Page 13: Scaling Online Social Networks (OSNs)](https://reader034.vdocument.in/reader034/viewer/2022042614/556aead3d8b42a86218b4d07/html5/thumbnails/13.jpg)
13
SPAR
Events● Nodes – Add/Remove● Edges – Add/Remove● Servers – Add/Remove
Motivation-Algorithms-Contribution-Evaluation
![Page 14: Scaling Online Social Networks (OSNs)](https://reader034.vdocument.in/reader034/viewer/2022042614/556aead3d8b42a86218b4d07/html5/thumbnails/14.jpg)
14
SPAR Algorithm
Motivation-Algorithms-Contribution-Evaluation
2
3
4
1
M1
M3
M2
5
5'
5
6'
1'
5'
6
Create Edge (1,6)
Master Node
Replica Node
![Page 15: Scaling Online Social Networks (OSNs)](https://reader034.vdocument.in/reader034/viewer/2022042614/556aead3d8b42a86218b4d07/html5/thumbnails/15.jpg)
15
SPAR Algorithm
Motivation-Algorithms-Contribution-Evaluation
2
3
4
1
M1
M3
M2
5
5'
5
6'
1'
5'
6
Create Edge (1,6)
C1: Create 6' in M1 Create 1' in M3
Master Node
Replica Node
6'
1'
![Page 16: Scaling Online Social Networks (OSNs)](https://reader034.vdocument.in/reader034/viewer/2022042614/556aead3d8b42a86218b4d07/html5/thumbnails/16.jpg)
16
SPAR Algorithm
Motivation-Algorithms-Contribution-Evaluation
2
3
4
1M1
M3
M2
5
6'
1'
5'
6
Create Edge (1,6)
C2: Move 1 to M3
Master Node
Replica Node
1'
4'
3'
2'
![Page 17: Scaling Online Social Networks (OSNs)](https://reader034.vdocument.in/reader034/viewer/2022042614/556aead3d8b42a86218b4d07/html5/thumbnails/17.jpg)
17
SPAR Algorithm
Motivation-Algorithms-Contribution-Evaluation
2
3
4
1
M1
M3
M2
5
5'
5
6'
1'
6
Create Edge (1,6)
C3: Move 6 to M1
Master Node
Replica Node
![Page 18: Scaling Online Social Networks (OSNs)](https://reader034.vdocument.in/reader034/viewer/2022042614/556aead3d8b42a86218b4d07/html5/thumbnails/18.jpg)
18
JA-BE-JA
● Distributed Partitioning Algorithm● K-way Partitioning● Load Balancing● Gossip Learning
Motivation-Algorithms-Contribution-Evaluation
![Page 19: Scaling Online Social Networks (OSNs)](https://reader034.vdocument.in/reader034/viewer/2022042614/556aead3d8b42a86218b4d07/html5/thumbnails/19.jpg)
19
JA-BE-JA - Policies
● Sampling– Local
● Select neighbors
– Random● Select from random
walk
– Hybrid● Local & Random
Motivation-Algorithms-Contribution-Evaluation
● Swapping– Energy Function
● Reach minimum
– Simulated Annealing● Escape from local
optima
Source: http://socialnetworking.lovetoknow.com/Growth_of_Online_Social_Networking_in_Business
![Page 20: Scaling Online Social Networks (OSNs)](https://reader034.vdocument.in/reader034/viewer/2022042614/556aead3d8b42a86218b4d07/html5/thumbnails/20.jpg)
20
Outline
● Motivation● Current Algorithms
– SPAR
– JA-BE-JA
● Contributions– Challenges
– Solution
● Evaluation & Conclusions
![Page 21: Scaling Online Social Networks (OSNs)](https://reader034.vdocument.in/reader034/viewer/2022042614/556aead3d8b42a86218b4d07/html5/thumbnails/21.jpg)
21
Challenges
Motivation-Algorithms-Contribution-Evaluation
SPAR
Global View requirement
Replication Overhead
Partition Manager→ Single Point of Failure
SPAR
![Page 22: Scaling Online Social Networks (OSNs)](https://reader034.vdocument.in/reader034/viewer/2022042614/556aead3d8b42a86218b4d07/html5/thumbnails/22.jpg)
22
Our Solution
Motivation-Algorithms-Contribution-Evaluation
SPAR&
JA-BE-JA
Global View requirement
Replication Overhead
Partition Manager→ Single Point of Failure Local View
Distributed PartitionManager
![Page 23: Scaling Online Social Networks (OSNs)](https://reader034.vdocument.in/reader034/viewer/2022042614/556aead3d8b42a86218b4d07/html5/thumbnails/23.jpg)
23
Our Solution (wait for it...)
Motivation-Algorithms-Contribution-Evaluation
SPAR Client Requests
Data StoreServers
![Page 24: Scaling Online Social Networks (OSNs)](https://reader034.vdocument.in/reader034/viewer/2022042614/556aead3d8b42a86218b4d07/html5/thumbnails/24.jpg)
24
Our Solution
Motivation-Algorithms-Contribution-Evaluation
JABEJA
SPAR&
JA-BE-JA
Client Requests
Data StoreServers
![Page 25: Scaling Online Social Networks (OSNs)](https://reader034.vdocument.in/reader034/viewer/2022042614/556aead3d8b42a86218b4d07/html5/thumbnails/25.jpg)
25
Outline
● Motivation● Current Algorithms
– SPAR
– JA-BE-JA
● Contributions– Challenges
– Solution
● Evaluation & Conclusions
![Page 26: Scaling Online Social Networks (OSNs)](https://reader034.vdocument.in/reader034/viewer/2022042614/556aead3d8b42a86218b4d07/html5/thumbnails/26.jpg)
26
Implementation
● SPAR● SPAR-JA
Motivation-Algorithms-Contribution-Evaluation
This is SPARJA!
![Page 27: Scaling Online Social Networks (OSNs)](https://reader034.vdocument.in/reader034/viewer/2022042614/556aead3d8b42a86218b4d07/html5/thumbnails/27.jpg)
27
Datasets
● Facebook Graphs
by Stanford Network Analysis Project
– #nodes: 150 #edges: ~3000
– #nodes: 224 #edges: ~6000
– #nodes: 786 #edges: ~60000
Source: http://snap.stanford.edu/
Motivation-Algorithms-Contribution-Evaluation
![Page 28: Scaling Online Social Networks (OSNs)](https://reader034.vdocument.in/reader034/viewer/2022042614/556aead3d8b42a86218b4d07/html5/thumbnails/28.jpg)
28
Datasets
● Synthesized Graphs– using our own Graph Generator
– #nodes: 1000, #degree: 10
Motivation-Algorithms-Contribution-Evaluation
ClusteredRandomized Highly Clustered
Graph Visualization Toolhttps://gephi.org/
![Page 29: Scaling Online Social Networks (OSNs)](https://reader034.vdocument.in/reader034/viewer/2022042614/556aead3d8b42a86218b4d07/html5/thumbnails/29.jpg)
29
ExperimentsReplication Overhead on Different Datasets
Motivation-Algorithms-Contribution-Evaluation
Synthesized Graphs10000 edges
synth-r: Randomizedsynth-c: Clusteredsynth-hc:
Highly Clustered
Facebook Graphsfcbk-1: ~3000 edgesfcbk-2: ~6000 edgesfcbk-3: ~60000 edges
#k-replicas: 0 (fault tolerance) #Servers: 4
![Page 30: Scaling Online Social Networks (OSNs)](https://reader034.vdocument.in/reader034/viewer/2022042614/556aead3d8b42a86218b4d07/html5/thumbnails/30.jpg)
30
ExperimentsReplication Overhead vs Replication Factor
Motivation-Algorithms-Contribution-Evaluation
K=0K=2
![Page 31: Scaling Online Social Networks (OSNs)](https://reader034.vdocument.in/reader034/viewer/2022042614/556aead3d8b42a86218b4d07/html5/thumbnails/31.jpg)
31
ExperimentsReplication Overhead on both algorithms
Motivation-Algorithms-Contribution-Evaluation
Fault ToleranceK=2
synth-hc: - Highly Clustered- Synthesized Graph- 10000 edges
![Page 32: Scaling Online Social Networks (OSNs)](https://reader034.vdocument.in/reader034/viewer/2022042614/556aead3d8b42a86218b4d07/html5/thumbnails/32.jpg)
32
ExperimentsReplication Overhead on both algorithms
Motivation-Algorithms-Contribution-Evaluation
Fault ToleranceK=2
fcbk-3: - 3rd facebook graph- 60,000 edges
![Page 33: Scaling Online Social Networks (OSNs)](https://reader034.vdocument.in/reader034/viewer/2022042614/556aead3d8b42a86218b4d07/html5/thumbnails/33.jpg)
33
Conclusions
● SPAR + JA-BE-JA = SPAR-JA– Highly clustered nodes
– Achieves fault tolerance 'by-default'
– Better than SPAR in case of high clusterization
● Future Work– More datasets
– Bigger datasets
Motivation-Algorithms-Contribution-Evaluation
![Page 34: Scaling Online Social Networks (OSNs)](https://reader034.vdocument.in/reader034/viewer/2022042614/556aead3d8b42a86218b4d07/html5/thumbnails/34.jpg)
Presented by: Maria Stylianou Coworker: Anis Uddin
Supervisor: Šarūnas Girdzijauskas
KTH - Royal Institute of TechnologyImplementation of Distributed Systems
December 6th, 2012
Scaling Online Social Networks (OSNs)