color-based object identification
DESCRIPTION
Color strategies for object identification Qasim Zaidi Marques Bostic State University of New York College of Optometry. Color-Based Object Identification. Can you identify which parts are sunny or shady? How? Why do you infer illuminant rather than material changes? - PowerPoint PPT PresentationTRANSCRIPT
Color strategies for object identification
Qasim ZaidiMarques Bostic
State University of New YorkCollege of Optometry
Color-Based Object Identification
Can you identify which parts are sunny or shady? How?Why do you infer illuminant rather than material changes?Can you identify similar foliages from sunny to shady parts?
Strategies for color-based object identification
Color Constancy: Constancy of subjective appearance by adapting to the illuminant ( Ives, Helson, Smithson & Zaidi etc.).
Constancy of Color Contrast/Differences
Inverse Optics Methods: Estimate spectrum of illuminant and infer reflectance spectra which are physical invariants (Maloney & Wandell, D’Zmura & Iverson, Brainard & Freeman etc.).
Neural algorithms: Based on invariants in cone-catches.
Physical invariants for color identification
For natural and man-made materials, changes in cone-absorptions of reflected lights for large sets are simply multiplicative, hence rank-orders and cone-contrasts are preserved (Dannemiller; Nascimento & Foster; Zaidi, Spehar & DeBonet).
Neural invariants for color identification
In post-receptoral cordinates, the color conversion is translational for L/(L+M), and multiplicative for S/(L+M) (Zaidi, Spehar & DeBonet).
HEURISTIC ALGORITHM FOR IDENTIFICATION
0
0.01
0.02
0.03
0.04
0.05
0.06
0.5 0.6 0.7 0.8
L/(L+M)
Equal Energy Light
0.5 0.6 0.7 0.8
L/(L+M)
Skylight
0.5 0.6 0.7 0.8
L/(L+M)
EE --> Skylight
Recovery of Lambertian Objects
L/(L+M)
Identify a subset of objects under equal-energy light with the superset under skylight: Assume that the affine transformation holds, and derive its parameters by matching patterns of chromaticities (Zaidi, 1998). This process should almost always lead to correct identification, which is easy to test.
Using parallel color difference vectors will lead to correct identifications for many but not all reflectances (Khang & Zaidi). Test this alternative by comparing distractors on lines parallel and orthogonal to the line joining the illuminants.
COLOR-DIFFERENCE/CONTRAST STRATEGY
Three of the objects are identical, but the fourth is made from a different paper. The two backgrounds are of the same material, but the
two illuminants have different spectra.
Which object is made from a different paper than the other three?
Exp 1: COLOR BASED OBJECT IDENTIFICATION
Given that neither of the discriminable objects on one side looked exactly like the objects on the other side, what strategy did you follow?
Number 1 is made from a different paper than the other three objects.
Stimuli
LegendEach data point is the result of 10 observations
O = side correct < 80%, i.e. observer cannot discriminate between standard and test under the same illuminant.
O = object correct ≥ 80%, i.e. observer identifies correct objects across illuminants.
O = object correct ≤ 20%, i.e. observer misidentifies the standard as the odd object.
O = 20% < object correct < 80%, i.e. no reliable identification across illuminants.
O = test object under bluish light
O = test object under yellowish light
Observer: RH – uninformed, experienced
Two other observers gave very similar results
RESULTS: COLOR BASED OBJECT IDENTIFICATION
RESULTS: COLOR BASED OBJECT IDENTIFICATION
RESULTS: COLOR BASED OBJECT IDENTIFICATION
RESULTS: COLOR BASED OBJECT IDENTIFICATION
Object Identification Results Summary•Object identification across illuminants is not perfect, even when tests can be discriminated from the standards.
•Because the standard object is a possible choice on every trial, the systematic misidentifications rule out color-constancy, inverse-optics and the affine-heuristic algorithm.
•Near the color of the standard, misidentified and correctly identified objects tend to fall on opposite sides, regardless of the color-directions of the distractors. This rules out the color-difference/contrast strategy.
•Can a simple perceptual strategy account for the performance results, similar to the strategy for lightness based object identification (Robillotto & Zaidi)?
Exp 2: RELATIVE COLOR CATEGORIZATION
Instructions: Compared to the standard object, is the test predominantly yellower, bluer, redder, or greener?
Observer: QZ – informed, experienced. Three other observers gave very similar results
Standard and test under the same light
Legend
The C O L O R of the circle corresponds to observer QZ’s judgment of the test object being predominately yellower, bluer, redder or greener
than the standard
O = no judgment could be made because the standard and test could not be discriminated
O = test object under bluish light
O = test object under yellowish light
RESULTS: RELATIVE COLOR CATEGORIZATION
RESULTS: RELATIVE COLOR CATEGORIZATION
RESULTS: RELATIVE COLOR CATEGORIZATION
RESULTS: RELATIVE COLOR CATEGORIZATION
Summary of Color Categorization Results
Standard and test under the same lights
* Tests separated into two relative color groups with respect to the standard. The division was similar under the two lights, and corresponded to clusters of correct and incorrect object identification.
Similarity based algorithm for object identification across illuminants
1.Identify the illuminant on the test (It) by finding the side on which the two objects have different colors.
2. Compare the colors of the backgrounds to judge the change in color from Is to It.
3. Pick the object most dissimilar to the others along the perceived It - Is color dimension as the odd object.
Predictions for tests under the blue/green light
Tests perceived as bluer/greener than the standard, will be picked correctly as the odd object. Tests perceived as yellower/redder than the standard, will not be picked as the odd object, unless they are yellower/redder than the standards in the yellow light. Opposite for tests under the yellow/red light
Similarity-based Predictions vs Results
I G M TOTAL
I 43 7 4 54
M 4 5 15 24
TOTAL 47 12 19 78
Correct predictions: 92% of identifications & 79% of misidentifications.
Observed
Predicted
From QZ’s relative colors to RH’s object identification.
I = correct identification, M = misidentification, G = neither
“Opportunistic” Bayesian Strategy Selection?
•Why do observers use color similarity for object identification despite the demonstrable absence of color constancy?•Why are more accurate alternatives not used despite sufficient information in natural scenes?•Is the visual system an “Opportunistic” Bayesian which includes the costs of additional computations in strategy selection?