CS639:DataManagementfor

DataScienceLecture26:Privacy

[slidesfromVitaly Shmatikov]

TheodorosRekatsinas

1

slide 2

Reading

• Dwork.“DifferentialPrivacy” (invitedtalkatICALP2006).

BasicSetting

xn

xn-1

!

x3

x2

x1

SanUsers(government,researchers,marketers,…)

query1

answer1

queryT

answerT

!DB=

randomcoins¢ ¢ ¢

slide 3

ExamplesofSanitizationMethods• Inputperturbation

• Addrandomnoisetodatabase,release

• Summarystatistics• Means,variances• Marginaltotals• Regressioncoefficients

• Outputperturbation• Summarystatisticswithnoise

• Interactiveversionsoftheabovemethods• AuditordecideswhichqueriesareOK,typeofnoise

slide 4

StrawmanDefinition• Assumex1,…,xn aredrawni.i.d.fromunknowndistribution

• Candidatedefinition:sanitizationissafeifitonlyrevealsthedistribution

• Impliedapproach:• Learnthedistribution• Releasedescriptionofdistributionorre-samplepoints

• Thisdefinitionistautological!• Estimateofdistributiondependsondata…whyisitsafe?

slide 5

Frequency in DB or frequency in underlying population?

BlendingintoaCrowd• Intuition:“Iamsafeinagroupofkormore”

• kvaries(3…6…100…10,000?)

• Manyvariationsontheme• Adversarywantspredicategsuchthat0<#{i|g(xi)=true}<k

• Why?• Privacyis“protectionfrombeingbroughttotheattentionofothers”[Gavison]

• Rarepropertyhelpsre-identifysomeone• Implicit:informationaboutalargegroupispublic

• E.g.,liverproblemsmoreprevalentamongdiabetics

slide 6

Clustering-BasedDefinitions• GivensanitizationS,lookatalldatabasesconsistentwithS

• Safeifnopredicateistrueforallconsistentdatabases

• k-anonymity• PartitionDintobins• Safeifeachbiniseitherempty,orcontainsatleastkelements

• Cellboundmethods• Releasemarginalsums

slide 7

brown blue S

blond [0,12] [0,12] 12brown [0,14] [0,16] 18S 14 16

brown blue S

blond 2 10 12brown 12 6 18S 14 16

IssueswithClustering

• Purelysyntacticdefinitionofprivacy• Whatadversarydoesthisapplyto?

• Doesnotconsideradversarieswithsideinformation• Doesnotconsiderprobability• Doesnotconsideradversarialalgorithmformakingdecisions(inference)

slide 8

“Bayesian”Adversaries• AdversaryoutputspointzÎ D• Score=1/fz iffz >0,0otherwise

• fzisthenumberofmatchingpointsinD

• SanitizationissafeifE(score)≤e• Procedure:

• Assumeyouknowadversary’spriordistributionoverdatabases

• Givenacandidateoutput,updatepriorconditionedonoutput(viaBayes’rule)

• Ifmaxz E(score|output)<e,thensafetorelease

slide 9

Issueswith“Bayesian”Privacy

• Restrictsthetypeofpredicatesadversarycanchoose• Mustknowpriordistribution

• Canoneschemeworkformanydistributions?• Sanitizerworksharderthanadversary

• Conditionalprobabilitiesdon’tconsiderpreviousiterations• Remembersimulatableauditing?

slide 10

ClassicalIntutionforPrivacy• “IfthereleaseofstatisticsSmakesitpossibletodeterminethevalue[ofprivateinformation]moreaccuratelythanispossiblewithoutaccesstoS,adisclosurehastakenplace.”[Dalenius1977]

• Privacymeansthatanythingthatcanbelearnedaboutarespondentfromthestatisticaldatabasecanbelearnedwithoutaccesstothedatabase

• Similartosemanticsecurityofencryption• Anythingabouttheplaintextthatcanbelearnedfromaciphertextcanbelearnedwithouttheciphertext

slide 11

ProblemswithClassicIntuition• Popularinterpretation:priorandposteriorviewsaboutanindividualshouldn’tchange“toomuch”

• Whatifmy(incorrect)prioristhateveryUTCSgraduatestudenthasthreearms?

• Howmuchis“toomuch?”• Can’tachievecryptographicallysmalllevelsofdisclosureandkeepthedatauseful

• Adversarialuserissupposed tolearnunpredictablethingsaboutthedatabase

slide 12

ImpossibilityResult• Privacy:forsomedefinitionof“privacybreach,”" distributionondatabases," adversariesA,$ A’suchthatPr(A(San)=breach)– Pr(A’()=breach)≤e

• Forreasonable“breach”,ifSan(DB)containsinformationaboutDB,thensomeadversarybreaksthisdefinition

• Example• ParisknowsthatTheois2inchestallerthantheaverageGreek• DBallowscomputingaverageheightofaGreek• ThisDBbreaksTheos’s privacyaccordingtothisdefinition…evenifhisrecordisnot inthedatabase!

slide 13

[Dwork]

(VeryInformal)ProofSketch• SupposeDBisuniformlyrandom

• EntropyI(DB;San(DB))>0

• “Breach”ispredictingapredicateg(DB)• Adversaryknowsr,H(r;San(DB))Å g(DB)

• Hisasuitablehashfunction,r=H(DB)

• Byitself,doesnotleakanythingaboutDB(why?)• TogetherwithSan(DB),revealsg(DB)(why?)

slide 14

DifferentialPrivacy(1)

xnxn-1!x3x2x1

San

query1answer1

queryTanswerT

!DB=

randomcoins¢ ¢ ¢

slide 15

◆ ExamplewithGreeksandTheoAdversarylearnsTheo’sheightevenifheisnotinthedatabase

◆ Intuition:“WhateverislearnedwouldbelearnedregardlessofwhetherornotTheoparticipates”Dual:Whateverisalreadyknown,situationwon’tgetworse

Adversary A

DifferentialPrivacy(2)

xnxn-1!0x2x1

San

query1answer1

queryTanswerT

!DB=

randomcoins¢ ¢ ¢ Adversary A

◆ Definen+1gamesGame0: Adv.interactswithSan(DB)Gamei: Adv.interactswithSan(DB-i);DB-i = (x1,…,xi-1,0,xi+1,…,xn)

GivenSandpriorp()onDB,definen+1posteriordistrib’s

slide 16

DifferentialPrivacy(3)

xnxn-1!0x2x1

San

query1answer1

queryTanswerT

!DB=

randomcoins¢ ¢ ¢ Adversary A

slide 17

Definition:Sanissafeif" priordistributionsp(¢)onDB," transcriptsS," i =1,…,n

StatDiff(p0(¢|S),pi(¢|S))≤e

Indistinguishability

xnxn-1!x3x2x1

San

query1answer1

queryTanswerT

!DB=

randomcoins¢ ¢ ¢

slide 18

transcriptS

xnxn-1!y3x2x1

San

query1answer1

queryTanswerT

!DB’=

randomcoins¢ ¢ ¢

transcriptS’

Differin1rowDistancebetweendistributionsisatmoste

WhichDistancetoUse?• Problem:emustbelarge

• Anytwodatabasesinducetranscriptsatdistance≤ne• Togetutility,neede >1/n

• Statisticaldifference1/nisnotmeaningful!• Example:releaserandompointindatabase

• San(x1,…,xn)=(j,xj )forrandomj

• Foreveryi,changingxi inducesstatisticaldifference1/n

• Butsomexi isrevealedwithprobability1

slide 19

?

Definition:Sanise-indistinguishable if" A," DB,DB’whichdifferin1row," setsoftranscriptsS

Adversary A

query1

answer1transcript

S

query1

answer1transcript

S’

Equivalently," S:p(San(DB)=S)p(San(DB’)=S) Î 1± e

p(San(DB)Î S)Î (1± e)p(San(DB’)Î S)

FormalizingIndistinguishability

slide 20

IndistinguishabilityÞ Diff.Privacy

Definition:Sanissafeif" priordistributionsp(¢)onDB," transcriptsS," i =1,…,n

StatDiff(p0(¢|S),pi(¢|S))≤e

ForeverySandDB,indistinguishabilityimplies

ThisimpliesStatDiff(p0(¢|S),pi(¢|S))≤e

slide 21

SensitivitywithLaplaceNoise

slide 22

DifferentialPrivacy:Summary

• Sangivese-differentialprivacyifforallvaluesofDBandMeandalltranscriptst:

slide 23

Pr [t]

Pr[ San (DB - Me) = t]Pr[ San (DB + Me) = t]

≤ ee » 1±e