npsychophysical experimental method to estimate image quality · npsychophysical experimental...
TRANSCRIPT
Digital image capture Digital image capture –– Camera Camera testing and quality assurancetesting and quality assurance
nnPsychophysical experimental Psychophysical experimental method to estimate image qualitymethod to estimate image quality
nn Hitoshi UrabeHitoshi UrabeFuji Photo Film Co.,Fuji Photo Film Co., Ltd. Ltd.
OutlineOutline
nn BackgroundBackgroundnn Requirements for psychophysical Requirements for psychophysical
experimental methods experimental methods nn Two stages procedure and a triplet Two stages procedure and a triplet
comparisoncomparisonnn ConclusionConclusionnn Future workFuture work
Psychophysical experimentsPsychophysical experiments
Method Stress
Categorical
Rank Order
Paired comparison
Stability
Low
High
Scalability
Low
High
Small
Large smaller
Magnitude
RequirementsRequirements
nn Enables a large number of samples to be Enables a large number of samples to be exampledexampled
nn Provides precise scalabilityProvides precise scalabilitynn Provides low observer stressProvides low observer stressnn Suitable for nonSuitable for non--expert observersexpert observersnn Provides high repeatability of the resultsProvides high repeatability of the results
Two stages procedureTwo stages procedure
Categorical session
Triplet comparison method session
Reduction of the number of the samples
Interval scaling
Two stages procedureTwo stages procedure
(1) The 1(1) The 1stst stage;stage;A categorical stage to reduce unnecessary A categorical stage to reduce unnecessary comparison (remove extremes) .comparison (remove extremes) .
(2) The 2(2) The 2ndnd stage;stage;A precise comparison stage, which A precise comparison stage, which enables to compare samples efficiently enables to compare samples efficiently by a triplet comparison.by a triplet comparison.
The The Number of CombinationNumber of Combination
AA
BB
CC
DDEE
GG
FF
7 samples 7 Combinations
DE,DG,EGDE,DG,EGDEGDEG
CD,CF,DFCD,CF,DFCDFCDF
BF,BG,FGBF,BG,FGBFGBFG
BC,BE,CEBC,BE,CEBCEBCE
AE,AF,EFAE,AF,EFAEFAEF
AC,AG,CGAC,AG,CGACGACG
AB,AD,BDAB,AD,BDABDABD
The Number of ObservationsThe Number of Observations
11761176392392494930030010010025251275127542542551513513511171172727
465465155155313121217777528528176176333336361212996666662222223737787826261313741741247247393910510535351515903903301301434317117157571919990990330330454521021070702121
Pair Pair Comp.Comp.
Triplet Triplet Comp.Comp.
No. of No. of SamplesSamples
Pair Pair Comp.Comp.
Triplet Triplet Comp.Comp.
No. of No. of SamplesSamples
ExperimentalExperimental
nn To examine the validity and To examine the validity and reliability of the proposed methodreliability of the proposed method
As a case studyAs a case study
nn To investigate the preferred skin To investigate the preferred skin colors on various mediacolors on various media
ObjectivesObjectives
Original portrait sceneOriginal portrait scene
Facial area of the standard portrait: L*=58.8, a*=17.0, b*=22.0
11stst Stage; ExperimentalStage; Experimental
nn 102 samples102 samplesnn 45 observers45 observersnn 3 categories 3 categories
(Favorable(Favorable, , Acceptable, UnacceptableAcceptable, Unacceptable))nn Viewing illuminant: fluorescent lamp Viewing illuminant: fluorescent lamp
for color evaluation (1000lx)for color evaluation (1000lx)
Test samplesTest samples
a*
b*
2yg
6yg
4yg
2y
4y
6y
2o
4o
6o
2r 4r 6r2g4g6g
2c
4c
6c
2b
4b
6b
2m
4m
6m
S SS
--33
+3+3
+6+6
+9+9
+12+12
1717 xx 66 == 102102 SamplesSamples
ΔΔL*L*
+4+4
+6+6
+2+2
Results Results
4R4R-3-3-33-331021022Y2Y+6+6+3+31414
4M4M+12+12-31-311011012B2B+6+6+4+41313
2YG2YG+6+6+4+41212
2G2G+3+3-2-225254M4M+9+9+4+41111
2R2R+3+3-2-224242M2M+3+3+6+61010
SS00-1-12323SS+3+3+6+699
2M2M+9+9-1-122222R2R+6+6+6+688
2R2R+9+9-1-121212Or2Or+9+9+6+677
2YG2YG+3+300202022GG+6+6+7+766
2Y2Y+9+900191922OrOr+6+6+7+755
4B4B+3+3+2+218182B2B+3+3+8+844
2Or2Or+3+3+2+2171722CyCy+3+3+8+833
2M2M+6+6+2+216162Cy2Cy+6+6+8+822
2YG2YG+9+9+2+21515SS+6+6+17+1711
HueL*ScoreOrderHueL*ScoreOrder
22ndnd Stage; ExperimentalStage; Experimental
nn 21 samples21 samples((Out of 102 samples)Out of 102 samples)nn 15 observers15 observers((including 4 experts)including 4 experts)nn 5 categories (1:bad 5 categories (1:bad ~~ 5:5:Excellent) Excellent) nn Paired comparison and Triplet Paired comparison and Triplet
comparisoncomparisonnn Repeated 5 times Repeated 5 times nn ScheffeScheffe’’ss methodmethod
Examples Examples
Triplet comparisonTriplet comparison
Observer:MIObserver:MI
Paired comparisonPaired comparison Triplet comparisonTriplet comparison
1
2
3
4
5
4
3
2
1
5
Times Times
S2c2o2r2m2b2y2g2yg4c4o4r4m4g4b4y4yg2k2w6w01-2-1201-2-121回目2回目3回目4回目5回目S2c2o2r2m2b2y2g2yg4c4o4r4m4g4b4y4yg2k2w6w SS2c2c2o2o2r2r2m2m2b2b2y2y2g2g2yg2yg4c4c4o4o4r4r4m4m4g4g4b4b4y4y4yg4yg2k2k2w2w6w6w01-2-1201-2-121回目2回目3回目4回目5回目
Bad Excellent Bad Excellent
Observer:MTObserver:MT
01-2-1201-2-121回目2回目3回目4回目5回目S2c2o2r2m2b2y2g2yg4c4o4r4m4g4b4y4yg2k2w6wS2c2o2r2m2b2y2g2yg4c4o4r4m4g4b4y4yg2k2w6w 01-2-1201-2-121回目2回目3回目4回目5回目S2c2o2r2m2b2y2g2yg4c4o4r4m4g4b4y4yg2k2w6wS2c2o2r2m2b2y2g2yg4c4o4r4m4g4b4y4yg2k2w6w
Paired comparisonPaired comparison Triplet comparisonTriplet comparison
1
2
3
4
5
4
3
2
1
5
Times Times
Bad Excellent Bad Excellent
Observer:TKObserver:TK
Paired comparisonPaired comparison Triplet comparisonTriplet comparison
1
2
3
4
5
4
3
2
1
5
Times Times
SS2c2c2o2o2r2r2m2m2b2b2y2y2g2g2yg2yg4c4c4o4o4r4r4m4m4g4g4b4b4y4y4yg4yg2k2k2w2w6w6w01-2-1201-2-121回目2回目3回目4回目5回2回目3回目4回目01-2-125回目01-2-121回目S2c2m2b2g4c4m4g4b6w2o2r2y2yg4o4r4y4yg2k2wS2c2o2r2m2b2y2g2yg4c4o4r4m4g4b4y4yg2k2w6w
Bad Excellent Bad Excellent
Psychophysical scalePsychophysical scale
Samples
Triplet
Paired
0
-2
+2
+1
-1
Interval scaleInterval scale
ObserverObserver’’s variance s variance
0
0.1
0.2
0.3
0.4
0.5
0.6
MI
KS
NT
CT
ST
MT
MN
MH
MH
EY
TY
YK
KK
YH
YS
Me
an
Observer
Va
ria
nce
TripletPairt
Scalability Scalability
Paired comparison Triplet comparison
Scale Range 1.51 2.29Standard Deviation 0.24 0.33
SR/SD 6.19 7.00
Correlation Correlation
y = 1.5768xR2 = 0.978
-1.5
-1
-0.5
0
0.5
1
1.5
-1.5 -1 -0.5 0 0.5 1 1.5
Paired comparison
Trip
let c
ompa
rison
CorrelationCorrelation
y = 0.9954xR2 = 0.9962-1.0
-0.5
0.0
0.5
1.0
-1.0 -0.5 0.0 0.5 1.0
Paired comparison
Tri
ple
t co
mp
ari
so
n
Thurston’s Case Ⅴ
Preferred skin colors Preferred skin colors
b*a*L*s2y2o2r2m2b2c2yg2g4y4o4r4m4yg4g4c4b2k2ws6w一般人専門家
O RG
C
B
M
YG
Y
O
+4
+2
a*
b*
Experimental time Experimental time
15 s
5 s
12 s
12 s
One set of One set of combinationcombination
For a pairFor a pair
Triplet comparison
Paired comparison
Required time for assessment
(In case of 20 samples)
Experimental time Experimental time
0
1
2
3
4
5
6
7
0 10 20 30 40 50 60
The number of samples
Paired comparison
Triplet comparison
Hours
ConclusionConclusion
TheThe triplet comparison reduced the assessment triplet comparison reduced the assessment time by more than 50% compared to the paired time by more than 50% compared to the paired comparison.comparison.
TheThe new psychophysical experimental method can new psychophysical experimental method can provide a useful procedure to estimate image provide a useful procedure to estimate image quality from the following points of view;quality from the following points of view;
・ Scalability of the method ・ Repeatability of the scale
・ Observer stress
Status of ISO 20462 :Status of ISO 20462 :Psychophysical experimental method to Psychophysical experimental method to
estimate image qualityestimate image quality
nn MultiMulti--part standard (ISO/TC42/WG18)part standard (ISO/TC42/WG18)nn Part 1 Part 1 –– General description of psychophysical General description of psychophysical
evaluation By Dr. evaluation By Dr. Keelan Keelan (Kodak)(Kodak)nn N4925_WD20462N4925_WD20462--1_E_.PDF1_E_.PDFnn Part 2 Part 2 –– Triplet comparison method By UrabeTriplet comparison method By Urabenn N4929_WD20462N4929_WD20462--2_E_.PDF2_E_.PDFnn Part 3 Part 3 –– Quality ruler method By Dr. Quality ruler method By Dr. KeelanKeelannn N4926_WD20462N4926_WD20462--3_E_.PDF3_E_.PDF