examining photo response non-uniformity for the comparison of cameras

NEDERLANDSFORENSISCHINSTITUUT

Nederlands Forensisch Instituut Laan van Ypenburg 6, 2497 GB Den Haag

Examining Photo Response Non-

Uniformity for the Comparison of

Cameras

Zeno Geradts PhD / Maarten van der Mark BS / Wiger van Houten MS

Partially funded by the European Commission within the project FIDIS

www.fidis.net


Overview• Introduction• Noise and PRNU sources• Comparison of denoising

methods• Application to YouTube videos• Performance• Statistics (Bayesian)• Conclusions


FIDIS

• Network of Excellence within the European Union, with 20 partners ranging from Universities, Privacy protection agencies and companies such as IBM and Microsoft

• www.fidis.net

http://www.fidis.net/


Introduction• Each image sensor has a unique

`fingerprint’, the PRNU pattern, that is detectable in all images the sensor produces

• This `fingerprint’ is used to establish the image origin


Noise and PRNU sources• Different types of noise:

• 2 Categories:I. Random noise

I. Temporally variableII. Statistical distributionsIII. Can be reduced by averaging multiple frames

II. Pattern noise (PRNU and FPN)I. Not temporally variable but systematic (spatial

variable)

• How do image sensors work?• CCD image sensors• CMOS Active Pixel Sensors


CCD image sensors• Basic building block (gate)• Silicon substrate • Apply positive voltage to the gate• Electron-hole pairs generated in depletion region are confined• Read out• Charge converted to voltage at the sense node


CMOS APS image sensors

• Cpd: photodiode capacitor

• M1: reset transistor• M2: source follower• M3: row select

transistor• M4: bias transistor


Noise and PRNU sources (2)

• The characteristic pattern extracted from the images contains 2 systematic contributions:• Multiplicative PRNU, depends on illumination• Fixed Pattern Noise (FPN), does not depend

on illumination

• Also: common components in the extracted pattern (e.g. CFA interpolation)


Noise and PRNU sources (3)

• FPN (general):• Crystal defects in the silicon

lattice introduced during growth• Impurities• The size of the detector/potential well• Contamination during fabrication• Non-uniform oxide/gate thickness• In CMOS: additional sources (each

transistor)


Noise and PRNU sources (4)• PRNU

The depth of the detector/potential well• Larger active area: more incident

photons• Non-uniform oxide layer: results in non-

uniform potential wells• Deeper potential well: more photons

absorbed (wavelength dependent)• In CMOS: additional sources


Image denoising – pattern extraction• Denoising algorithms do not discriminate

between noise and image details. There is always a tradeoff.

• PRNU is a nonperiodic discontinuous signal

• Pattern = Image – F(Image) , F is the denoising filter

• Correlate the patterns from `questioned’ videos with reference patterns


Old method• Gaussian smoothing filter• Advantages:

• Simple• Very fast

• Disadvantages:• Distorts edge integrity• Image residue left behind in

the pattern


New method• Wavelet based denoising

filter [Lukas et al]*• Disadvantages

• Slower• Diadic images only

• Advantages• Preserves edges (edge

detection)• Spatial adaptive • Works really well

*Digital Camera Identification from Sensor Pattern Noise (2005) – Lukas, Fridrich, Goljan


YouTube (1)• Accepts large amount of input

formats:Xvid, DivX, WMV, 3GP, …

• Downloadable as H.264/MPEG-4 AVC with e.g. keepvid.com

• Maximum resolution (H.264): 480x360

• Aspect ratio generally does not change


YouTube (2)• When the resolution of the video

uploaded to YouTube is smaller than 480x360, generally no resolution change occurs (exceptions)

• When the resolution of the video exceeds 480x360, the resolution is changed to 480x360 or lower (depending on the aspect ratio)e.g. 640x480 480x360

640x360 480x270


In practice – natural video• Record 30 seconds of natural

video, recorded with XVID/WMV• Upload this video to YouTube• Download the video with

keepvid.com• Extract individual frames• Estimate noise pattern


In practice – flatfield video• If natural video downloaded has a

resolution lower than 480x360, record the flatfield video (RAW) with the same resolution, use these frames

• Otherwise (e.g. 640x480), record flatfield video in native resolution, and upload (download) them to (from) YouTube• Alternative: resize flatfield video to match

the dimension of the YouTube video (e.g. 640x480 480x360)


Performance (1)• Works really well when natural video is

recorded in native resolution, both with XVID and WMV9 (Messenger) with large range of codec settings, even for shorter samples

• Works reasonably well when aspect ratio has been changed

• Problematic with very low resolution (Vodafone: 176x144)

• Does not always work when the video was binned during recording (e.g. native resolution of 640x480, recorded in 320x240x)


Performance (2) – Creative Live! IM Webcam

• Problem: need to set two parameters for extracting the patterns

• These parameters depend on a large amount of variables: • Content• Compression• Codec• Resolution• etc.

• Only from empirical data; impossible in casework


Comparison – Creative Live! IM, 640x480 native, recorded in 352x288 • Old method

(σ=0.6, threshold 5, 4x4 averaging):Camera Correlatio

n

11.5 0.2048

11.6 0.1893

11.4 0.1669

11.1 0.1273

11.2 0.1191

11.7 0.1061

New Method (σ_n=4.5, σ_f=4.5):

Camera Correlation

11.4 0.0531

11.1 0.0271

11.2 0.003

11.7 -0.0106

11.5 -0.0237

11.6 -0.0263


Logitech Communicate STX (1)• Logitech :

640x480 (native), XVID q4

• Logitech : 640x480 (native), variable XVID quality


Logitech Communicate STX (2)• Logitech :

320x240, XVID q4

• Logitech : 320x240, variable XVID quality


Test yourself sourceforge.net search PRNU


Likelihood ratioLikelihood ratio

•Comparing patterns•Value of the evidence:


Likelihood ratioLikelihood ratio• In practice:

• No overlapping histograms• We do not know the origin of the questioned

image


Likelihood ratioLikelihood ratio•To artificially find the pdf of Hd at Hp

we use an estimator•Example: add-constant estimatorSuppose you look at the colours of passing cars:• 1 blue car, 1 red car, and 1 green car• Q: what is the chance the next car is blue?• A: 1/3? Or smaller?• A: (1+1)/7 for blue; 1/7 for a new colour



•More complicated estimators available, but the essence is the same: reserve a probability for unseen species, based upon the data we encountered so far

•Simple Good-Turing approach



•SGT: problems:• Car colours (green, blue, red, etc) is

discrete, but some colours exist that can be seen as both (blue or green?)

• We are considering correlation values, continuous data. We have to divide the values into `species’ ourselves


Likelihood ratio, recapLikelihood ratio, recap


Conclusion (1)Conclusion (1)

•We understand the origins of the pattern noise that is used to perform device identification

•We understand which random noise sources are present

•A likelihood ratio can be found, but there are some unanswered questions before this approach can be used


Conclusions (2)• It is possible to identify the source

video camera based on the PRNU pattern, even after the video has been uploaded to YouTube

• The new method, although computationally more intensive, performs much better


Conclusions (3)• Limitations: (very) low resolution

videos, changed aspect ratios, ‘subsampled’ recordings

• No way to find the ideal parameter directly from the video or frames: this is a problem when the abovementioned limitations are met


Future research

• Using the methods on databases of child pornography to link cameras

• Improvement of algorithms• Validation in casework


Questions ?

• [email protected]

mailto:[email protected]

examining photo response non-uniformity for the comparison of cameras

Documents