1

GraphSC: Parallel Secure Computation Made Easy

Kartik Nayak

With Xiao Shaun Wang, Stratis Ioannidis, Udi Weinsberg, Nina

Taft, Elaine Shi

2

Users

Data

Data

Privacy concern!

Data Mining Engine

Data Model

Data Mining on User Data

3

Graph representing social connections

Graph representing professional connections

Compute user’s influence in both

circles

Companies Computing on Private Data

4

Companies want to run

machine learning algorithms

Users/Companies do NOT want to reveal data

Can we enable this in practice?

Cryptography to the rescue:

Secure Multiparty Computation

Ensures that we learn only the outcome

6

Key Challenges

Generic Solutions1

Lot of work improving individual algorithmsDeparture from one-at-a-time approach

7

Key Challenges

Convert Program to

Run on Secure Computation

(Cost of obliviousness)

2

8

Key Challenges

Parallelizability3There’s a lot of data – maintain benefits of parallelism in the insecure settingWith cryptography, expensive computation

9

Key Contributions

10

Key Contributions

Generic Framework for

“Graph-parallel”

AlgorithmsPregel

by

PageRank

Matrix Factorization using gradient descent

Risk Minimization using ADMMAnd many more

Matrix Factorization using ALS

Challenge:Generic

Solutions

11

Key Contributions

Efficiently Convert Graph-parallel Programs to

Oblivious ProgramsTotal work blowup is O(log |

V|)

Blowup for naïve solution: O(|V|) for sparse graphs

Challenge:Convert

program to run on Secure

Computation

12

Key Contributions

Maintain Parallelizability

Depth of the computation is O(log |V|)

Matrix Factorization:4K ratings, 32 threads [NIWJTB’13]1.4 hours

Challenge:Parallelizability

< 4 mins

13

1

2

3

Efficiently Convert to Oblivious Programs

Maintain Parallelizability

Generic Framework for Graph-parallel Algorithms

Key Contributions

14

function bs(val, s, t) mid = (s + t) / 2; if (val < mem[mid]) bs(val, 0, mid) else bs(val, mid+1, t)

Programmer’s favorite model

Cryptographer’s favorite model

15

Programmer’s model:

Programs

Oblivious Programs

Cryptographer’s model: Circuits

Intuitively,Program traces should not depend on

input data

16

function bs(val, s, t) mid = (s + t) / 2; if (val < mem[mid]) bs(val, 0, mid) else bs(val, mid+1, t)

Programmer’s favorite model

Cryptographer’s favorite model

17

Programmer’s model:

Programs

Oblivious Programs

Cryptographer’s model: Circuits

Intuitively,Program traces should not depend on

input data

Easy

Hard

18

Achieving Parallelism

Goal: Low Depth

Circuits

Oblivious Parallel RAM [BCP’14]

Polylogarithmic Blowup:

Not practical

GraphSC: O(log |V|) blowup

19

Pregel by

“Graph-parallel” algorithms

[LGKB’10, GLGBG’12, MABDHLC’10, ZCF’10]

20

Graph-parallel Algorithms

A

B

C

D

1

2

4

5

1

1

1

2

47

1

0

1Scatter: Send data to

edges

Gather: Aggregate data from edges

Apply: Perform some computation

21

Obliviousness of Graph-parallel Algorithms

Do not reveal edge/vertex data

Do not reveal structure of the graph

Naïve Solution: O(|V|2)

A

B

C

D1

1

1

2

4 Our Solution:

O(|E| log|V|)

7

1

0

1

22

Oblivious Gather – Key Trick

3

4

1

2

Oblivious Gather – Key Trick

Oblivious Sort with (v, isVertex)

Single pass

Sort: O(|E| log |V|)Single pass: O(|E|)

Oblivious Gather: (|E| log |V|)

Gather in clear: O(|E|)

24

Scatter

Complexity of Our Algorithms

Gather

Apply

Sequential Insecure (Total Work)

Parallel Oblivious (Total Work)

ParallelOblivious

(Parallel Time)

O(|E|)

O(|V|)

O(|E| log |V|)

O(|E|)

O(log |V|)

O(1)

NaïveOblivious(Total Work)

O(|V|2)

O(|E|)

25

Algorithms on GraphSCHistogram computationPageRankMatrix Factorization using gradient descentMatrix Factorization using alternating least squares

Bellman-Ford shortest path

Bipartite matching

Parallel empirical risk minimization through alternating direction method of multipliers (ADMM)

Pregel by

26

Experimental Setup

……Cloud 1

(Garblers)Cloud 2

(Evaluators)Two Scenarios:1. LAN2. Across Data

Centers (WAN)

27

Key Evaluation Results

Histogram

Input Size

1K – 0.5M

Parallel Time (32

processors)4 sec – 34 minPageRank (1

iteration)Matrix

Factorization (1

iteration)

Using GD

Using ALS

4K – 128K

20 sec – 15.5 min1K –

32K47 sec – 34 min

64 – 4K 2 min – 2.35 hours

28

Max: 16K ratings (64x smaller data) [NIWJTB’13]

Running at Scale

Matrix Factorization using gradient descent: 1M ratings, 6K users, 4K movies [KBV’09]

7 machine cluster, 128 processors, 525 GB RAM

Time taken: ~13 hours(1 iteration)

4K ratings, 32 threads1.4 hours < 4 mins

We used only 7 machines!

13 hours -> few mins by using more machines

29

Across Data Centers

Page RankGarblers: OregonEvaluators: N. Virginia

B/W provisioned: 2 Gbps

Time reduces linearly with increasing processors

30

GraphSC is a parallel secure computation framework for Graph-

parallel algorithms

www.oblivm.com

Thank You!kartik@cs.umd.edu

Conclusion