wcpm 1 chang-tsun li department of computer science university of warwick uk image clustering based...
Post on 19-Jan-2018
214 Views
Preview:
DESCRIPTION
TRANSCRIPT
WCPM 1
Chang-Tsun Li
Department of Computer Science
University of Warwick
UK
Image Clustering Based on Camera Fingerprints
WCPM 2
Digital Image Acquisition Process
Scene Post-Processing
Lens
Sensor CFA InterpolationCFA Photo
CFA: Colour Filter Array
R G
G B
Bayer CFA R G
G B
mapping of CFA to sensor pixels
R G
G B
R G
G B
R G
G B
R G
G B
R G
G B
WCPM 3
What is Camera Fingerprint• Lens aberration
• Sensor pattern noise
• Colour filter array (CFA) interpolation artefacts
• Camera response function
• Quantisation table of JPEG compression
Scene Post-Processing
Lens
Sensor CFA InterpolationCFA Photo
WCPM 4
Camera Fingerprint for Multimedia ForensicsMultimedia Forensics: The use of “fingerprints” left in images by the imaging devices for
•source device identification
•source device linking
•content integrity verification
•image classification
WCPM 5
What is Sensor Pattern Noise Sensor Pattern Noise (SPN) is the noise left in the
images by the sensors of digital imaging devices such as cameras, camcorders and scanners.
SPN is mainly caused by – manufacturing imperfection and – different sensitivity of pixels to light due to the
inhomogeneity of silicon wafers. Sensors made from the same silicon wafer possess
unique SPN because of the non-uniform imperfection.
SPN can differentiate cameras of the same model.
WCPM 6
“Traditional” SPN Extraction Method Lukáš et al’s model for SPN extraction (IEEE TIFS
2006)
– I is the original image– I’ is the low-pass filtered version of I by the Weiner
filter applied in the wavelet transform domain– n is the extracted SPN
SPN is the high-frequency component of the image.
)(_' IfilterWeinerI
),('),( jiIjiIn
WCPM 7
Interference from Scene Details Scene details, e.g., brick walls, tree leaves, or other kinds of
textures, contribute to the high-frequency components of images.
a contaminated SPN
natural image
SPN
a clean SPN
WCPM 8
SPN Enhancement at Warwick• C.-T. Li, "Source Camera Identification Using Enhanced Sensor
Pattern Noise," IEEE Trans. on Information Forensics and Security, June 2010
• C.-T. Li and Y. Li, "Color-Decoupled Photo Response Non-Uniformity for Digital Image Forensics," IEEE Trans. on Circuits and Systems for Video Technology, 2012
• X. Lin and C.-T. Li, "Preprocessing Reference Sensor Pattern Noise via Spectrum Equalization," IEEE Trans. on Information Forensics and Security, 2016 .
• X. Lin and C.-T. Li, "Enhancing Sensor Pattern Noise via Filtering Distortion Removal," IEEE Signal Processing Letter, accepted for publication in 2016
WCPM 9
Image Classification/Clustering
Scenario: A forensic investigator •has a large set of images taken by an unknown number of unknown digital cameras and •wishes to cluster those images into a number of classes, each including the images acquired by the same camera.
Each data point represents one image Each cluster present one unknown device
WCPM 10
Challenges Facing Image Classification
The forensic investigator does not have the cameras that have taken the images to generate reference SNPs for comparison.
No prior knowledge about the number and types of the imaging devices are available.
With a large dataset, exhaustive fingerprint comparison is computationally prohibitive.
Given the shear number of images, analysing each image in its full size is computationally infeasible.
WCPM 11
Image Classification – a MRF ApproachStep 1. Extract and enhance the fingerprint of each block cropped from the images
Step 2. Establish a similarity matrix ρ for a Focus Set of M images
Step 3. Train the classifier based on the similarity matrix ρ. For each fingerprint i, which is treated as a random variable
3.1. Assign a unique random class label
3.2. Calculate a reference similarity (i.e., a “soft” threshold)
3.3. Establish a membership committee (neighbourhood)
3.4. Update the class label iteratively based on the information from the membership committee until there are no changes of class labels to any SPN throughout a entire iteration
Step 4. Classify the rest of the dataset using the classifier
WCPM 12
Establishing Similarity Matrix To establish an M × M similarity matrix ρ, the similarity between any two enhanced SPNs i and j in the Focus Set is calculated using
},...,3,2,1{, , )()(
),( Mjinnnn
nnnnji
jjii
jjii
i , j 1 2 3 4 …. M
1 1.00
2 1.00
3 1.00
4 1.00:: 1.00
M 1.00
WCPM 13
Classifier Training• Each fingerprint (SPN) is treated as a random variable. 3.1. Assign a unique random class label to each SPN 3.2. Calculate a reference similarity r Normally intra-class similarities > inter-class similarities.
•A similarity less than r indicates that the two images are taken by different devices, otherwise by the same device.
similarity Similarity
inter-class similarity intra-class similarity
μ1 r μ2
221
r
WCPM 14
3.3. Establish a membership committee For each SPN i , a membership committee Ci with c SPN
members from the focus set that are most similar to i is established.
Classifier Training
vvvvvv
×
WCPM 15
• fi : class label of SPN i
• C• ρ(i, Ci ) is the similarities between
SPN i and the members of Ci, i.e.,
3.4. Update the class label iteratively according to p(fi |ρ(i, Ci ), Li) until there is no change of class label to any SPN in x consecutive iterations
Classifier Training
• p(fi |ρ(i, Ci ), Li): probability of assigning fi given the conditions
• ri: reference similarity (“soft” threshold) of I
}|{}{ ijii CjffL
}|),({),( ii CjjiCi
)]),,(,(exp[1)),,(|( iiiii
iii LCifUZ
LCifp
)]),,(,(exp[
ii Lf
iiiii LCifUZ
WCPM 16
similarity Similarity
inter-class similarity intra-class similarity
μ1 r μ2
The combination of the s(.) and ρ(.) says, • a penalty (i.e. positive value) will be incurred
if ρ(i,j) > r and a different label than fi is to be assigned to i or
if ρ(i,j) < r and the same label as fj is to be assigned to i• a reward (i.e. negative value) will be given
if ρ(i,j) < r and a different label than fi is to be assigned to i or
if ρ(i,j) > r and the same label as fj is to be assigned to i
Objective Function / Cost Function
iCj
ijiiiii rjiffsLCifU ),(),()),,(,(
ji
jiji ff
ffffs
if , 1 if , 1
),(
WCPM 17
Image Classification
The centroids of the image clusters provided by the classifier training process at the end of Step 3.4 are used to classify the images.
To classify an image x, we compare the similarity of its SPN to the centroid of each identified cluster and classify it to the class with its centroid closest to the image.
WCPM 18
Clustering in progress …..
Classifier Training - Simulation
Initial label configure: Each pattern is assigned an unique label / colour
WCPM 19
Final Classification
WCPM 20
Experimental Results
c
Block Size
256 × 256 256 × 512 512 × 512
Focus set size (M) Focus set size (M) Focus set size (M)
120 300 120 300 120 300M-1
M/2
M/3
M/4
M/5
Table 1. Classification error rate. c is the size of the membership committee.
c: the size of the membership committeeM: the size of the focus set
8.889
8.333
8.333
8.333
8.333
4.000
4.000
4.000
4.000
4.000
3. 778
2.333
2.333
2.333
3. 778
1.333
1.333
1.333
1.333
1.222
1.444
1.444
1.444
1.444
1.444
1.444
1.444
1.444
1.556
1.444
Misclassification rates: 1200 images taken by six cameras, each taking 200. Class identification stops when there are no class label changes throughout an iteration.
A misclassification rate in the range (1.2 ~ 1.6) is likely to be the best the system can achieve.
WCPM 21
Conclusions
Multimedia forensics using “fingerprint” left in the images by the imaging devices has emerged as a new area of research in the last few years.
Sensor pattern noise (SPN) is one of the most promising types of fingerprint.
The “traditional” SPN extraction method is unable to cope with the interference of scene details.
The proposed classifier is feasible, but is unable to classify images without clean SPNs provided by the proposed SPN enhancer.
top related