Blind VisionShai Avidan, Moshe Butman

Yuval Schwartz

Ethic Problems

• Proliferation of surveillance cameras leads to privacy concerns

• Protection by all-the-way video encryption• Problems:

– What happens if a virus attacks the server?– What if the service provider is not trusted?

• Common Examples:– Web face detection– Government agency compares private images to suspects– CPU shortage– Blind OCR

Introduction

• Face Detection:– Alice: a set of images privacy is a must– Bob: a face detection algorithm a secret algorithm

• Demands:– Alice: will learn nothing about Bob’s detector– Bob: will learn nothing about Alice’s images nor the detector’s

result

• Apply secure multi-party techniques to vision algorithms to enforce the demands– Computationally intensive– Domain specific constaints new schemes secure enough (?)

Agenda

• Secure multi-party computation– Secure two-party problem and algorithm– Oblivious Transfer– Specific:

• Millionaire problem• Secure dot-product

• Secure Classifier– Complexity and Efficiency

• New Scheme to Accelerate:– Image Hashing using HoG

• Experiments• Results

Secure two-party problem1

• Problem description:– F – polytime function– X – Alice’s input– Y – Bob’s input– F(X,Y) – output– Demands:

• Alice won’t know Y• Bob won’t know X• Alice and/or Bob will know F(X,Y)

1. A.C. Yao, How to generate and exchange secrets, 27th FOCS, pp. 162-167, 1986

Yao’s Protocol

• General Idea:– Imagine F as a boolean circuit C (has boolean

gates)– A method to run the circuit:

• without revealing the input wires values

• The output must be exposed

Boolean Gate

Boolean Gate1

23

• Demands: Can’t reveal input bits and if it’s a middle gate then also the output bit

Wire00 wK11 wK

1K

2K

),( g ),(

3gK

• Problem: Seams impossible to calculate the gate with values unknown

),(3

gK

Boolean Gate

Boolean Gate1

23

Computation Table:

1K

2K

Output Wire W3Input Wire W2Input Wire W1

01K01K11K11K

02K12K02K12K

)0,0(3gK

)1,0(3gK

)0,1(3gK

)1,1(3gK

Example: OR Gate

Output Wire W3Input Wire W2Input Wire W1

01K01K11K11K

02K12K02K12K

03K13K13K13K

Output Wire W3Input Wire W2Input Wire W1

01K01K11K11K

02K12K02K12K

03K13K13K13K

Garbled Wire W3

))(( 030

201

KEEKK

))(( 131

201

KEEKK

))(( 130

211

KEEKK

))(( 131

211

KEEKK

),(321

21,

g

KKKEEKK

Garbled Circuit

G1E(E(K)

K

K

G2E(E(K)

K

K

G3E(E(K)

K

K

G4E(E(K)

K

K

G5E(E(K)

K

K

G6E(E(K)

K

K

G7E(E(K)

K

K

G8E(E(K)

K

K

G9E(E(K)

K

K

GBE(E(K)

K

K

GCE(E(K)

K

K

GDE(E(K)

K

K

GAE(E(K)

K

K

11XK

22XK

33XK

44XK

55XK

16YK

27YK

38YK

49YK

510YK

1),(11

YXFK

2),(12

YXFK

3),(13

YXFK

4),(14

YXFK

Output Decryption

Table

Yao’s Protocol

11XK

22XK

33XK

44XK

5

5XK

16YK

27YK

38YK

49YK

510YK

1),(11

YXFK

2),(12

YXFK

3),(13

YXFK

4),(14

YXFK

Problem:

• Alice doesn’t have the input map

• Bob can’t give the whole input map

Solution:

• Oblivious Transfer

Oblivious Transfer2,3

2. M.O. Rabin, How to exchange secrets by oblivious transfer, Tech. Memo TR-81, Aiken Computation Laboratory, 1984

3. S. Even, O. Goldreich and A. Lempel, A Randomized Protocol for Signing Contracts, Communications of the ACM 28, pp. 637-647, 1985

Oblivious Transfer

S0

S1

K1

K0

KK0

K

K’

S0

Secure Dot-Product

• Input:– Alice:

– Bob:

• Output: – Demands: Bob won’t know x and Alice won’t know y

• Idea:– Break the result of the dot product to a+b,

where a is known only to Alice and b is known only to Bob.

yxyx T

LFy

LFx ]255,0[ix}1,0,1{iy

Secure Dot-Product

)2,17,13(x )1,1,1( y

)4,8,3(b

2821713 yx

iiii bxya

,...)10,9,...,0,1,2,3(0 a

,...)9,8,...,5,6,7,8(1 a

,...)7,6,5,4(2 a

bxya OT

100 a

OT

91 a

)6,9,10( a

OT

62 a

15b

13a

Secure Dot-Product

• Security:– From Alice to Bob: the use of OT hides xi

– From Bob to Alice: b as a random vector hides y

• Complexity:L – the dimensionality of x and y

)(LO

Secure Millionaire

• Idea: represent the two numbers in binary format and scan it from the MSB to the LSB with a map made by Bob and Alice traversing the map

><=

Secure Millionaire

Input: Alice has a number x = 855 = 1101010111

Bob has a number y = 810 = 1100101010

Output: Alice and Bob find out if x > y

1. Bob defines three states:AAlice has a larger number

BBob has a larger number

UUndecided

2. For MSB, Bob constructs a 2-entry lookup table z(n)

yn=0yn=1

xn=0UB

xn=1AU

Alice uses with xn as her index to obtain s(n)=z(n)(xn)

FxFy

21OT

Secure Millionaire

3. For each i=n-1,…,1: (a) Bob constructs a 6-entry lookup table z(i) that is indexed by s(i+1) and xi:

UA

BB

AA

BU

BB

AA

yi=1yi=0

0)1( i

i xAs

0)1( i

i xBs

0)1( i

i xUs

1)1( i

i xAs

1)1( i

i xBs

1)1( i

i xUs

(b) Alice uses with s(i+1) and xi as her indices to obtain s(i)=Z(i)(s(i+1),xi)6

1OT

4. Bob sends Alice the meaning of the three states of s(1) Alice knows which number is larger (and can send the result to Bob)

x = 855 = 1101010111y = 810 = 1100101010

Secure Millionaire

• Security:– From Alice to Bob: Alice uses OT so Bob can’t

learn nothing about x– From Bob to Alice: the values of the state s are

represented using random numbers for each bit

• Complexity: n – number of bits in x and y

)(nO

Secure ClassifierInput: Alice has input test pattern

Bob has a strong classifier of the form

Output: Alice has the result H(x) and nothing else

Bob learns nothing about the test pattern x

N

nn xhsignxH

1

)()(

otherwise

yxxh

n

nnT

nn

)(

otherwises

yxsxh

nn

nnT

nnn

)(

otherwises

yxsxh

nn

nnT

nnn

)(

Secure Dot-Product

Secure Millionaire

N

nn xh

1

)(

N

nnss

1

><=

Secure Classifier

• Security:– Secure dot-product– s as a random vector for obfuscating the real parameters– Alice can learn the number of week classifiers

• Complexity: O(NLK)N – number of weak classifiersL – dimensionality of the test vector xK – number of bits in the dot-product xTyn

• Problem: a few seconds to a few minutes to classify a detection window

Accelerating Blind Vision

• Reduce number of operations taken for OT

• Bob reveals stripped-down classifier to Alice

• One-way hash functions:– Hides Alice’s Image– Still let Bob correctly classify the patterns– Classifier won’t work on hashed space

HoG

• Usefull in a variety of object recognition and detection applications

• Parameters for hash function:– Destroys the spatial order of the

pixels– Destroys the absolute values of the

pixels– Coarsely binned

HoG

HoG

• 18 bins

• Build an image for every bin (18 response images) where a pixel’s intensity represents the bin value

• Scrambling the order of pixels destroys spatial relationship between the HoGs

Experiments

• Secure Viola-Jones type face detector:– Small number of critical visual features from a

larger set– Cascade rejectors– Adjustments were made– Alice and Bob are allowed to a decide after

every level of the cascade

Results

• A single 24x24 detection window can be classified in several minutes using all cascade levels

• Usually the first two levels are enough to reject a pattern

• Accelerating: using scrambled HoGs and neural network to analyse – several seconds to process a single 240x320 image (rejects 90%)

Results

Flaws

• No mathematical security proofs

More Reading…

• S. Avidan, M. Butman, Efficient Methods for Privacy Preserving Face Detection, Advances in Neural Information Systems (NIPS 18), 2006

• A.C. Yao, How to generate and exchange secrets, 27th FOCS, pp. 162-167, 1986

Questions