multimedia data hidingminwu/public_paper/thesis/fpo_apr01.pdf · 2 fpo 4/01 electrical engineering...

11

Multimedia Data HidingMultimedia Data Hiding

Min Wu

Dept. of Electrical Engineering

Committee: Profs. B.Liu, P.Ramadge, S.Kulkarni

FPO 4/01

Electrical Engineering Dept.Princeton University

2

IntroductionIntroduction

! Digital Watermarking / Multimedia Data Hiding– Hide secondary data in digital image/video/audio/3D

! Uses of hidden data:– ownership verification, alteration detection – access control, annotation, side info. delivery

! Issues and challenges:– imperceptibility, robustness & security,

capacity– tradeoff between the conflicting

requirements

RobustnessRobustness

CapacityCapacity

ImperceptibilityImperceptibility

FPO 4/01


3

General FrameworkGeneral Framework

marked media(w/ hidden data)

embedembeddata to be data to be hiddenhidden

original media

compresscompress

process / process / attackattack

extractextract

play/ record/…play/ record/…extracted extracted datadata

playerplayer

101101 …101101 …“Hello, World”“Hello, World”

101101 …101101 …“Hello, World”“Hello, World”

FPO 4/01


4

Key Elements of Data HidingKey Elements of Data Hiding

! Perceptual model

! Embedding one bit

! Multiple bits

! Uneven embedding capacity

! Robustness and security

! What data to embed

Physical Layer“how to embed one or multiple bits?”

Upper Layers

uneven capacity equalization

error correction

security

……

22

FPO 4/01


5

Thesis OutlineThesis Outline

" Fundamental issues and solutions # embedding strategy classification and capacity issues# handling uneven capacity# modulation/multiplexing techniques for hiding multiple bits

" Algorithm and system designs# binary images# image authentication# video copy/access control and fingerprinting# applications in video communication

" Attacks and countermeasures# “innocent tools” – block replacement and double-capturing# countermeasure against rotation/scale/translation# robustness & security analysis on SDMI audio watermarking

FPO 4/01


6

Algorithm and System DesignsAlgorithm and System Designs

" Demonstrating solutions to fundamental issues

FPO 4/01


7

Data Hiding in Binary ImageData Hiding in Binary Image

! A simple yet important class of images– scanned documents, drawings, signatures

! Challenges– little room for “invisible” changes– uneven distribution of changeable pixels

Clinton electronically signed Electronic Signatures Act- Yahoo News 6/30/00 http://www.whitehouse.gov/

media/gif/bil.gif as of 7/00

E-PAD (InterLink Electronics)

FPO 4/01


8

ExampleExample--1: “Signature in Signature”1: “Signature in Signature”

– Annotating digitized signature with content info. of the signed document (Finkelstein - Princeton U.)

33

FPO 4/01


9

Our ApproachOur Approach

! Block-based pixel-domain method– hide a fixed number of bits in each block– extract hidden data without the use of original copy

! Three issues– determine which pixels to flip for invisibility

– embed data in each block using flippable pixels

– handle uneven embedding capacity via shuffling

Robustness is not a major requirement for authentication and annotation applications.

FPO 4/01


10

Preserve Visual QualityPreserve Visual Quality

! Assign flippability score to each pixel– determine how noticeable the flipping of a pixel is– based on smoothness and connectivity– Hierarchical

! Sort pixels in each block according to the scores– flip high-score pixels with high priority

(a) (b)

FPO 4/01


11

Embedding MechanismEmbedding Mechanism

! Extracting data without original image– hard to directly encode data in flippable pixels

# flippability may change after encoding

! Our approach– manipulate flippable pixels to enforce block-based property

# enforce the total number of black pixels to be odd/even to hide 1 bit / block, or use more general mapping

# incorporate quantization or tolerance zone for robustness

# of black pixel per blk 2kQ (2k+1)Q (2k+2)Q (2k+3)Qodd-even mapping

lookup table mapping0 1 0 1

… 0 1 1 0 …FPO 4/01


12

Pixels with high flippability score are shown in the images.

Shuffling for Binary ImageShuffling for Binary Image

! Uneven distribution of flippable pixels– most are on rugged

boundary

! Embedding rate (per block)

– variable: need side info.– constant: require larger blk

! Random shuffling equalizes distribution– embed more bits– enhance security 0 5 10 15 20 25 30 35 40 45 50

0

0.05

0.1

0.15

0.2

0.25

embeddble coeff. # per block (signature img)

porti

on o

f blo

cks

before shuffleafter shuffle

Important !Important !

image size 288x48, red block size 16x16

44

FPO 4/01


13

changeable pixel/coeff.

unchangeable pixel/coeff. S balls in totaln = pS blue balls

Analysis of ShufflingAnalysis of Shuffling

– Mean follows hypergeometric distribution

. . .q balls N = S/q blocks

pick w/o replacement

m r ~ # of blocks each having r blue balls out of q balls

FPO 4/01


14

0 5 10 15 20 25 30 35 400

0.05

0.1

0.15

0.2

0.25

# of flippable pixels per block (signature img)

porti

on o

f blo

cks

(x 1

00%

)

before shuffsimulation meansimulation stdanalytic meananalytic stdbefore shuffle

std after shuffle

mean after shuffle

Compare Analysis with Simulation for ShufflingCompare Analysis with Simulation for Shuffling

Simulation: 1000 indep. random shuff.

q = 16 x 16

S = 288 x 48

N = S/q = 18 x 3

p = 5.45%

std after shufflemean after shufflebefore shuffle

0.00100.00105.56x10-3 %5.81x10-3 %5.56%m2/N (2nd bin)

03.79x10-40 %7.77x10-4 %1.85%m1/N (1st bin)

09.78x10-50 %5.16x10-5 %20.37%m0/N (0th bin)

simulationanalysissimulationanalysis

FPO 4/01


15

ExampleExample--1: “Signature in Signature”1: “Signature in Signature”

– Annotating digitized signature with content info. of the signed document (Finkelstein - Princeton U.)

Each block is 320-pixel large, 1bit / blk.

FPO 4/01


16

ExampleExample--2: Document Authentication2: Document Authentication

– Embed pre-determined pattern or content features beforehand– Verify hidden data’s integrity to decide on authenticity

(f)

alter(a)

(b)

(g)

after alteration

(e)

(c)

(d)

55

FPO 4/01


17

Robustness vs. CapacityRobustness vs. Capacity! Blind/non-coherent detection ~ original copy unavailable

! Single robustness-capacity setting– over-estimates and/or under-estimates actual noise– not all embedded data are equally important

RobustnessRobustness

CapacityCapacity

ImperceptibilityImperceptibility

stronger noisenoise weaker

-15 -10 -5 0 5 10 15 200

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

10log10(E2/σ2) (dB)

Cap

acity

C (b

its/c

h. u

se)

Capacity of Type-I (host=10E) and Type-II AWGN ch. (wmk MSE E2)

Type-I (C-i C-o, blind detection)Type-II (D-i D-o)

-4 -3 -2 -1 0 10

0.02

0.04

0.06

0.08

0.1

grayscale/color image/video

FPO 4/01


18

Experimental ResultsExperimental Results

Video Examples

Level-1: high capacity Level-2: high robustnessembed. rate robustness embed. rate robustness Notes

60-frame 352 x240 flowergarden sequence

640 bits(91 char.)

132 bits(18 char.)

660-frame 352 x240 concatenatedsequence

3032 bitsMPEG-2 4.5Mbps;frame dropping 1266 bits

MPEG-2 1.5Mbps;frame dropping

alsoembedcontrolinfo.

avg. chunk size = 6 frames

FPO 4/01


19

Video Example Video Example

– 1st & 30th Mpeg4.5Mbps frame of original, marked, and their luminance difference– human visual model for imperceptibility: protect smooth areas and sharp edges

FPO 4/01


20

Robust Video Data HidingRobust Video Data Hiding

! Embedding domain

– partition video into segments of similar consecutive frames– for each frame of a segment, embed same data in block-DCT domain– embed segment index to detect frame jitter

! Multi-level embedding– light processing ⇒ data extractable from just a few frames– severe processing ⇒ extractable by processing more frames

! Uneven embedding capacity – within a frame: constant embedding rate per region & shuffling– between frames: embed # of hidden bit per frame as side info.

! Modulation/Multiplexing techniques– “TDMA”, “CDMA”, orthogonal/bi-orthogonal modulation

embed b i & (i mod M)

seg. i seg. i+1

embed b i+1 & (i+1 mod M)

66

FPO 4/01


21

Attacks & CountermeasuresAttacks & Countermeasures

FPO 4/01


22

Watermark Attacks: What and Why?Watermark Attacks: What and Why?

! Attacks: intentionally obliterate watermarks– remove a robust watermark– make watermark undetectable (e.g., miss synchronization)– uncertainty in detection (e.g., multiple ownership claims)– forge a valid (fragile) watermark– bypass watermark detector

! Why study attacks?– identify weaknesses– propose improvement– understand pros and

limitation of tech. solution

To win each campaign, a generalTo win each campaign, a generalTo win each campaign, a generalTo win each campaign, a generalTo win each campaign, a generalTo win each campaign, a generalTo win each campaign, a generalTo win each campaign, a generalshould know both his troop and should know both his troop and should know both his troop and should know both his troop and should know both his troop and should know both his troop and should know both his troop and should know both his troop and the opponent’s as well as possible.the opponent’s as well as possible.the opponent’s as well as possible.the opponent’s as well as possible.the opponent’s as well as possible.the opponent’s as well as possible.the opponent’s as well as possible.the opponent’s as well as possible.

---- Sun Sun TzuTzu, , The Art of War, The Art of War, 500 B.C.500 B.C.

FPO 4/01


23

“Innocent Tools” Used by Attackers“Innocent Tools” Used by Attackers

! Recovery of lost blocks– for resilient multimedia transmission of JPEG/MPEG– good quality by edge-directed interpolation: Jung et al; Zeng-Liu

! Remove robust watermark by block replacement

edge estimation

edge-directed interpolation

FPO 4/01


24

! Attack effective on block-DCT based spread-spectrum watermark

marked original (no distortion)JPEG 10% after proposed attack

JPEG 10% w/o distort Interp.

w/ orig 34.96 138.51 6.30 w/o orig 12.40 19.32 4.52

512x512 lennaThreshold: 3 ~ 6

claimed high robustness&quality by fine tuning wmk strength for each region

77

FPO 4/01


25

SSecure ecure DDigital igital MMusic usic IInitiative Challengenitiative Challenge

! International consortium ~ 180+companies/organizations– currently pursuing watermark based solution for access and

copy control on digital music

! Public challenge ( 9/15-10/8/2000 ) – attacks on four robust watermark technologies

! Non-traditional research values– reveal real industrial problem and state-of-art technologies– present an emulated rivalry environment for better

understanding on audio watermarking– lead to a few research problems

FPO 4/01


26

SDMI Challenge SetupSDMI Challenge Setup• Obtained From SDMI• Job for “Attackers”• Black Box (unknown)

“Watermark “Watermark Found”Found”DetectDetect

Any Marked Audio

EmbedEmbed

WatermarkWatermark(special signal)(special signal)Sample-1

(original)Sample-2 (marked)

“Watermark “Watermark NOTNOT Found”Found”AttackAttack DetectDetect

Sample-3 (marked) Sample-4

(attacked)

GOALGOALGOALGOALGOALGOALGOALGOAL

FPO 4/01


27

Observation on One SDMI WatermarkObservation on One SDMI Watermark

– Difference between original and marked samples given by SDMI

FPO 4/01


28

Learning from SDMI ChallengeLearning from SDMI Challenge

! Our successful attacks– blind attacks: warping, jittering– attacks based on studying orig.-marked pairs

# deliberate filtering / subtraction / randomization

! Noteworthy issues– duality between embedding and attacks

– secrecy of embedding can’t rely on orig. being unknown

– double-watermarking used by SDMI# robust wmk ⇒ should resist processing/attacks

# fragile wmk ⇒ indicate audio experience compression

– attacks and countermeasures on forging fragile wmk# relate to watermark-based authentication

88

FPO 4/01


29

Is Watermark Useful?Is Watermark Useful?! Not an answer to all

– our attacks pointed out weaknesses of specific proposals and demonstrated general approaches

! For copy/access control– hard to get complete solution with technology alone

# business model, pricing model, etc.

– improved watermark tech. could be part of the solution# make attack non-trivial and keep honest people honest

! Other applications– detecting alteration

# digital camera/camcorder; digitized signature/ binary doc.

– convey side information# for performance enhancement or additional funtionality

FPO 4/01


30

SummarySummary

FPO 4/01


31

Summary & ConclusionSummary & Conclusion

! Data hiding in digital multimedia for a variety of purposes, involving multiple disciplines

! Tradeoff among many criterions

! Important to think both as designer and as attacker

! Data hiding in market– digital cameras with authentication watermark module– plug-in for image editors– video watermark proposals for DVD copy control– on-going SDMI effort for digital music

– “Digital Rights Management (DRM)” for multimedia data

FPO 4/01


32

Other Data Hiding Works in ThesisOther Data Hiding Works in Thesis! Watermark-based image/video authentication, attack &

countermeasures– hide auth. data via look-up table in quantized coeff. (ICIP’98 &’99)– double capturing attack and countermeasure (Asilomar’99)

! Rotation/Scale/Translation resilient watermarking (w/ NECI)– add spread-spectrum wmk in log-polar of FFT magnitude (Trans. IP’01,

SPIE’00)

! Data hiding for video communication (w/ P. Yin) – real-time video transcoding via downsizing

# send subblock motion for better visual quality

– error concealment# protect motion vectors by embedding parity bits

99

FPO 4/01


33

Publication ListPublication ListFundamental Issues1. M. Wu, B. Liu: “Data Hiding in Images and Videos: Part I – Fundamental Issues and Solutions”, draft, to

be submitted to IEEE Trans. on Circuits & Systems for Video Technology, Feb. 2001.

2. M. Wu, H. Yu, A. Gelman: “Multi-level Data Hiding for Digital Image and Video”, SPIE’99.

3. M. Wu, B. Liu: “Digital Watermarking Using Shuffling”, IEEE ICIP'99.

Designs1. C-Y. Lin, M. Wu, Y-M. Lui, J.A. Bloom, M.L. Miller, I.J. Cox: “Rotation, Scale, and Translation Resilient

Public Watermarking for Images,” to appear in IEEE Transactions on Image Processing, May 2001.

2. M. Wu, B. Liu: “Data Hiding in Binary Images”, submitted to IEEE Trans. on Multimedia, Apr. 2001.

3. M. Wu, H. Yu, B. Liu: “Data Hiding in Images and Videos: Part II – Designs and Applications”, draft, to be submitted to IEEE Trans. on Circuits & Systems for Video Technology, Feb. 2001.

4. M. Wu, B. Liu: “Data Hiding for Image and Video Authentication”, to be submitted to IEEE Trans. on Image Processing, Jan. 2001

5. M. Wu, E. Tang, B. Liu: “Data Hiding in Digital Binary Image”, IEEE ICME'00.

6. M. Wu, H. Yu: “Video Access Control via Multi-level Data Hiding”, IEEE ICME'00.

7. P. Yin, M. Wu, B. Liu: “Video Transcoding by Reducing Spatial Resolution”, IEEE ICIP’00.

8. C-Y. Lin, M. Wu, J.A. Bloom, M.L. Miller, I.J. Cox, and Y-M. Lui: “Rotation, Scale, and Translation Resilient Public Watermarking for Images,” SPIE’2000.

www.ee.princeton.edu/~minwu/research.html

FPO 4/01


34

(list of design papers - cont’d)

9. M. Wu, B. Liu: “Watermarking for Image Authentication”, ICIP'98.

10. P. Yin, M. Wu, B. Liu: “Error Concealment for MPEG Video Over Internet”, submitted to ICIP’01.

Attacks & Countermeasures 1. M. Wu, S. Craver, E. Felten, B. Liu: “Analysis of Attacks on SDMI Audio Watermarks”, to appear in IEEE

ICASSP'01.

2. S. Craver, P. McGregor, M. Wu, B. Liu, A. Stubblefield, B. Swartzlander, D.S. Wallach, D. Dean, E.W. Felten: “Reading Between the Lines: Lessons from the SDMI Challenge”, to appear in 4th Info. Hiding Workshop, 2001.

3. M. Wu, B. Liu, “Attacks on Digital Watermarks”, Asilomar’99.

Non-watermark Works on Video (not included in thesis)1. M. Wu, R. Joyce, H-S. Wong, L. Guan, S-Y. Kung: “Dynamic Resource Allocation Via Video Content and

Short-term Traffic Statistics”, to appear in IEEE Trans. on Multimedia, special issues on multimedia over IP, June 2001.

2. M. Wu, R. Joyce, S-Y. Kung: “Dynamic Resource Allocation Via Video Content and Short-term Traffic Statistics”, ICIP ’00, invited paper.

3. H-S. Wong, M. Wu, R. Joyce, L. Guan, S-Y. Kung: “A Neural Network Approach For Predicting Network Resource Requirement in Video Transmission,” IEEE Pacific Rim Conference on Multimedia (PCM’00).

4. M. Wu, W. Wolf, B. Liu, "An Algorithm of Wipe Detection", IEEE ICIP'98.

FPO 4/01


35

Acknowledgement– Prof. Bede Liu (advisor)– Dr. Wenjun Zeng (HVS and error concealment)– Prof. Adam Finkelstein, Ed Tang, Mishella Yoshi (binary image)– Dr. Heather Yu (multilevel data hiding)– Peng Yin (transcoding and error concealment)– Scott Craver, Prof. Ed Felten (SDMI attacks)– Drs. I. Cox, M. Miller, J. Bloom, H. Stone (data hiding & RST wmk)

Questions? Questions? Comments? Comments? Welcome!Welcome!

multimedia data hidingminwu/public_paper/thesis/fpo_apr01.pdf · 2 fpo 4/01 electrical engineering...

Documents