Privacy for Free:Posterior Samplingand Stochastic Gradient Monte Carlo

Y.-X. Wang, S. Fienberg, and A. Smola

[Fast Differentially Private Matrix Factorization @ RecSys2015]

1

2

(ε, δ) A

X X’

3

Pr[A(X) À S] f exp(✏) Pr[A(X®

) À S] + �

Aε

4

ε

A(X) XX

X X’

A S

A(X’) A(X)

Pr[A(X) À S] f exp(✏) Pr[A(X®

) À S], ≈S À Range(A)

S

Pr[A(X) = S]

Pr[A(X®) = S]

5

X R RPr[R]

q(X, R) X R

X R

Δq(2εΔq)

q(X, R) − | f(X) − R |

∫ exp(ε q(X, R)) dPr(R)

exp(✏q(X,R)) Pr[R]

6

π(θ) θ

l(x | θ) θ x

X={x} θ

B l(x | θ) 4B

l(x | θ) 2B

Ç

✓ Ì Pr[✓X] ◊ exp

�≥x

l(✓x)�⇡(✓)

7

f(U,V)= 12

P(x,y)2D (r

xy

� u>x

vy

)2+ �

2 (kUk2F

+kVk2F

)

8

y�Y x�X

(x, y)�D�R

�U�V

rxy

uxvy

l(Rij | θ) ∝ −(Rij − ui⊤ vj)2

9

PrX

PrX Pr’X L1

δA

(ε, (1+eε) δ)

✓

t+1

} ✓

t

* ⌘

t

⇠(⇡(✓) +

N

⌧

≥⌧

i

(l(x

i

✓)⇡+ Normal(0, ⌘

t

)

Bibliography I

C. C. Aggarwal and P. S. Yu, editors.Privacy-Preserving Data Mining: Models and Algorithms.Springer, 2008.

J. A. Calandrino, A. Kilzer, A. Narayanan, E. W. Felten, and V. Shmatikov.You might also like: Privacy risks of collaborative filtering.In IEEE Sympo. on Security and Privacy, pages 231–246, 2011.

C. Dwork, F. McSherry, K. Nissim, and A. Smith.Calibrating noise to sensitivity in private data analysis.In Proc. of the 3rd Theory of Cryptography Conference, pages 265–284, 2006.[LNCS 3876].

Y. Koren.Collaborative filtering with temporal dynamics.In Proc. of the 15th ACM SIGKDD Int’l Conf. on Knowledge Discovery and DataMining, pages 447–455, 2009.

Z. Liu, Y.-X. Wang, and A. J. Smola.Fast differentially private matrix factorization.In Proc. of the 9th ACM Conf. on Recommender Systems, 2015.

10

Bibliography II

F. McSherry and I. Mironov.Differentially private recommender systems: Building privacy into the netflix prizecontenders.In Proc. of the 15th ACM SIGKDD Int’l Conf. on Knowledge Discovery and DataMining, pages 627–635, 2009.

F. McSherry and K. Talwar.Mechanism design via differential privacy.In Proc. of the 48th IEEE Sympo. on Foundations of Computer Science, pages 94–103,2007.

R. Salakhutdinov and A. Mnih.Probabilistic matrix factorization.In Advances in Neural Information Processing Systems 20, pages 1257–1264, 2008.

Y.-X. Wang, S. E. Fienberg, and A. Smola.Privacy for free: Posterior sampling and stochastic gradient monte carlo.In Proc. of the 32nd Int’l Conf. on Machine Learning, 2015.

11