brief announcement: practical summation via gossip
DESCRIPTION
Brief Announcement: Practical Summation via Gossip. Wesley W. Terpstra, Christof Leng, Alejandro P. Buchmann Databases and Distributed Systems Group Technische Universität Darmstadt Germany. Sum calculation in peer-to-peer. Input: every peer has a value Output: (at least) one peer knows - PowerPoint PPT PresentationTRANSCRIPT
www.dvs1.informatik.tu-darmstadt.de
Brief Announcement:Practical Summation via Gossip
Wesley W. Terpstra, Christof Leng, Alejandro P. Buchmann
Databases and Distributed Systems Group
Technische Universität Darmstadt
Germany
2
DA
TA
BA
SE
S A
ND
DIS
TR
IBU
TE
D S
YS
TE
MS
TE
CH
NIS
CH
E U
NIV
ER
SIT
ÄT
DA
RM
ST
AD
T
Sum calculation in peer-to-peer
Input: every peer has a value
Output: (at least) one peer knows
Useful in computing many global statistics: Network size Average utilization Load balance (standard deviation) Churn (rate of peer replacement) Size of stored data
For our system, BubbleStorm, we compute degi(p)
€
x pp∈P
∑
€
x p
3
DA
TA
BA
SE
S A
ND
DIS
TR
IBU
TE
D S
YS
TE
MS
TE
CH
NIS
CH
E U
NIV
ER
SIT
ÄT
DA
RM
ST
AD
T
Build on an existing solution
Approaches can be compared by Message rounds (latency) Total messages (bandwidth) Parameters: system size (n), accuracy ()
We improve the Push-Sum algorithm for practical use
Rounds Messages
Push-Sum (2003, FOCS)
Sample&Collide (2006)
Random Tour (2006)
Comp&Spread (2006)
€
logn + log1
ε
€
n logn + log1
ε
⎛
⎝ ⎜
⎞
⎠ ⎟
€
logn +1
εn
€
1
εn logn
€
n +1
ε 2
€
1
ε 2n
€
1
ε 2log2 n
€
1
ε 2 n log2 n
4
DA
TA
BA
SE
S A
ND
DIS
TR
IBU
TE
D S
YS
TE
MS
TE
CH
NIS
CH
E U
NIV
ER
SIT
ÄT
DA
RM
ST
AD
T
Analogy: Measuring a lake’s volume
5
DA
TA
BA
SE
S A
ND
DIS
TR
IBU
TE
D S
YS
TE
MS
TE
CH
NIS
CH
E U
NIV
ER
SIT
ÄT
DA
RM
ST
AD
T
Push-Sum visualized
6
DA
TA
BA
SE
S A
ND
DIS
TR
IBU
TE
D S
YS
TE
MS
TE
CH
NIS
CH
E U
NIV
ER
SIT
ÄT
DA
RM
ST
AD
T
Stationary Distribution (Steady State)
Perturbations of equilibrium do not affect water/fish ratio
Equilibrium: edges carry the same water and fish in both directions peers have water and fish proportional to degree and clock
7
DA
TA
BA
SE
S A
ND
DIS
TR
IBU
TE
D S
YS
TE
MS
TE
CH
NIS
CH
E U
NIV
ER
SIT
ÄT
DA
RM
ST
AD
T
Improvement: Big Fish eat smaller fish
8
DA
TA
BA
SE
S A
ND
DIS
TR
IBU
TE
D S
YS
TE
MS
TE
CH
NIS
CH
E U
NIV
ER
SIT
ÄT
DA
RM
ST
AD
T
Fish eating in the Network
9
DA
TA
BA
SE
S A
ND
DIS
TR
IBU
TE
D S
YS
TE
MS
TE
CH
NIS
CH
E U
NIV
ER
SIT
ÄT
DA
RM
ST
AD
T
Stationary Distribution (Steady State)
10
DA
TA
BA
SE
S A
ND
DIS
TR
IBU
TE
D S
YS
TE
MS
TE
CH
NIS
CH
E U
NIV
ER
SIT
ÄT
DA
RM
ST
AD
T
Other improvements
Round switching Once the result is accurate “enough”, restart Provides a running estimate on network statistics
Compensate for message loss
Prevent adding two of the most aggressive fish
Save bandwidth for multiple measurements
11
DA
TA
BA
SE
S A
ND
DIS
TR
IBU
TE
D S
YS
TE
MS
TE
CH
NIS
CH
E U
NIV
ER
SIT
ÄT
DA
RM
ST
AD
T
Synchrony
Kempe et al. prove correctness with synchronous model, but conjecture that it works asynchronously We validate this claim by simulation
1 million peers, 5s gossip interval, find network size:
0
10
20
30
40
50
60
27:00 29:00 31:00 33:00 35:00
Logarithmic size estimate
Time (mm:ss)
MaximumStd dev.
Minimum
12
DA
TA
BA
SE
S A
ND
DIS
TR
IBU
TE
D S
YS
TE
MS
TE
CH
NIS
CH
E U
NIV
ER
SIT
ÄT
DA
RM
ST
AD
T
Open Problem
Push-Sum is very vulnerable to attack Any peer can completely change the result This is largely due to the problem statement (sum!)
Simplistic prevention (bounds) easily defeated Introduce too few of the largest fish type too large Switch rounds prematurely too small & unstable
What is a useful adversary model for summation?
www.dvs1.informatik.tu-darmstadt.de
?Questions
Thanks for listening!