from colour to perception to neuroscience arne valberg norwegian university of science and...

From Colour to Perception to Neuroscience

Arne Valberg

Norwegian University of Science and Technology

Section of Biophysics and Medical Physics

Trondheim, Norway

Newton and optical Newton and optical phenomenaphenomena

The nature of coloursThe nature of colours(Leonardo da Vinci about 1500)(Leonardo da Vinci about 1500)

””There are six simple colours…. White is the There are six simple colours…. White is the first among the simple colours, yellow the first among the simple colours, yellow the second, green the third, blue the fourth, red second, green the third, blue the fourth, red the fifth, and black the sixth”the fifth, and black the sixth”

Leonardo da Vinci: Traktat von der Malerei. Jena, Leonardo da Vinci: Traktat von der Malerei. Jena, Diederichs 1925, p.85. Diederichs 1925, p.85.

The hue circle

Colour perception

” ” In the eye there are three types of nerve In the eye there are three types of nerve fibres. Stimulation of the first one excites the fibres. Stimulation of the first one excites the sensation of red, stimulation of thre second sensation of red, stimulation of thre second the sensation of green, stimulation of the the sensation of green, stimulation of the third the sensation of violet”third the sensation of violet”

and he continuesand he continues

”…(the essence of Young’s hypothesis is) that ”…(the essence of Young’s hypothesis is) that the sensations of colour are imagined as the sensations of colour are imagined as composed of three mutually and completely composed of three mutually and completely independent processes in th neural independent processes in th neural substrate.” substrate.”

HHermann von Helmholtz, 1860/1911ermann von Helmholtz, 1860/1911

Young-Helmholtz’ view on Young-Helmholtz’ view on colour visioncolour vision

Relative spectral sensitivity of receptors in the eye

Distribution of cone receptors.(no rods and no S-cones in the central fovea).

λmax = 570 nm (yellow), 545 nm (yellow-green), and 445 nm (violet)

Figur fra Valberg, 2005

Colour measurement as relative receptor excitations L, M, and S

jj

James Clerk Maxwell (1831 – 1879)

and the colour top

Maxwell on colour visionMaxwell on colour vision ””All vision is colour vision, for it is only by All vision is colour vision, for it is only by

obesrving differences of colour that we obesrving differences of colour that we distinguish the forms of objects. I include distinguish the forms of objects. I include differences of brightness or shade among differences of brightness or shade among differences of colour.”differences of colour.”

”… ”… it is not necessary to specify any given it is not necessary to specify any given colours as typical for these primary sensations. colours as typical for these primary sensations. … any of three colours might have been chosen, … any of three colours might have been chosen, provided that white resulted from their provided that white resulted from their combination in proper proportions”combination in proper proportions”

James Clerk Maxwell (1856)James Clerk Maxwell (1856)

The visual spectrum (380 to 780 nm)

Symmetric hue circle CIE chromaticity diagram

Edwin H. Land (1909 – 1991)

Registration of nerve pulses from a nerve cell with a microelectrode

[imp/s]

MC-cells respond to contours PC-cells respond to wavelength distributions

Threshold- and suprathreshold responses ofThreshold- and suprathreshold responses of

cone-opponent cellscone-opponent cells

and how to simulate themand how to simulate them

Data base:Data base: Single cell recordings from the LGN of Single cell recordings from the LGN of macaque monkeys (done at Max Planck Institute macaque monkeys (done at Max Planck Institute for biophysical Chemistry, Göttingen (D)) . for biophysical Chemistry, Göttingen (D)) .

Collaborators:Collaborators: Barry B. Lee, Thorstein Seim, and Barry B. Lee, Thorstein Seim, and Jo Tryti.Jo Tryti.

Barry B. LeeBarry B. Lee

Stimulus and cone-opponent PC-cell responses

Spectral threshold contrast sensitivity of 4 Spectral threshold contrast sensitivity of 4 opponent cells and one non-opponent MC-cell opponent cells and one non-opponent MC-cell

that resemble human psychophysicsthat resemble human psychophysics

Thorstein SeimThorstein Seim

How do we arrive at a model for colour scaling?

A possible (theoretical) combination of the different cone opponent cell responses along orthogonal axes.

(response to the white adapting stimulus subtracted)

M – LI&D cells

S – LI/D cells

L – MI&D cells

M – SI cells

W

10 5 0 5 10 15

15

10

5

0

5

10

15

G

R

Y

B

N N

NMUNSELLVALUE 5

N

Rela

tive

ce

ll re

spo

nse

s

Relative cell responses

W

Response

Respo

nse

OSA-UCS (Y = 30%)OSA-UCS (Y = 30%)colour scalig represented by a theoretical colour scalig represented by a theoretical combination of opponent cell responsescombination of opponent cell responses

How does the brain do it?How does the brain do it?

(Top down)(Top down)

V2 V3 V3A

V3

V2

V1

V3A

V5A

V4

V5

A

B

Visual areas of the cortex

Contour (PC, KC)

Movement and direction of movement (MC)

Orientation,eye dominance

Depth, 3Dcolour

Parallel pathwaysParallel pathways

The representation of colours in V2The representation of colours in V2

The hue circle

Colour perception

The endThe end

The modelling of opponent cell responses The modelling of opponent cell responses (retina and LGN)(retina and LGN)

Opponent combination (difference) of cone signals,Opponent combination (difference) of cone signals,

for instance Vfor instance VMM and V and VLL::

NNM–LM–L= (A= (AMMVVMM – A – ALLVVLL )+ N )+ Noo for a ’M-L’ cell (I- and D-for a ’M-L’ cell (I- and D-cells)cells)

NNL–ML–M = (B = (BLLVVLL – B – BMMVVM M )+ N)+ Noo for a ’L-M’ cell (I- and D- for a ’L-M’ cell (I- and D-cells)cells)

N:N: parvocellular cell (PC-cell) response in impulses/s parvocellular cell (PC-cell) response in impulses/s V:V: receptor signals (hyperpolarization of cones) receptor signals (hyperpolarization of cones) A A andand B: B: weighting constants (A weighting constants (AMM>A>ALL for I-cells and for I-cells and

AAMM<A<ALL for D-cells, etc.) for D-cells, etc.) NNoo: cell activity without stimulation: cell activity without stimulation

SummarySummary● ● Perception. Opponent unique colours red-green, Perception. Opponent unique colours red-green,

yellow-blue, white-black. Mechanisms unknown.yellow-blue, white-black. Mechanisms unknown.● ● The spectrumThe spectrum● ● Colour matches. Photoreceptors L, M, and S Colour matches. Photoreceptors L, M, and S

(retina).(retina).● ● Opponent ganglion cells: ’L-M’, ’M-L’, ’M-S’, ’S-Opponent ganglion cells: ’L-M’, ’M-L’, ’M-S’, ’S-

L’. Incr. & Decr. cells (retina, LGN).L’. Incr. & Decr. cells (retina, LGN).● ● Simulation of cell responses (retina, Simulation of cell responses (retina,

LGN)/model (V1?). Scaling.LGN)/model (V1?). Scaling.● ”● ”Bottom up” (retina, LGN) versus ”top down” Bottom up” (retina, LGN) versus ”top down”

(V4, V2...).(V4, V2...).● ● Unique colours and surround effects Unique colours and surround effects

(simultaneous contrast, adaptation, ”colour (simultaneous contrast, adaptation, ”colour constancy”) unexplained.constancy”) unexplained.

The two main roadsThe two main roads Leonardo da Vinci (ca. 1500Leonardo da Vinci (ca. 1500)) Isaac Newton (ca. 1670)Isaac Newton (ca. 1670) Thomas Young and George Palmer (1770 – Thomas Young and George Palmer (1770 –

1800)1800) James Clerk Maxwell (ca. 1860)James Clerk Maxwell (ca. 1860) Hermann von Helmholtz (ca. 1860)Hermann von Helmholtz (ca. 1860) Hermann Grassmann (1850)Hermann Grassmann (1850) Ewald Hering (ca. 1870)Ewald Hering (ca. 1870) Wilhelm Ostwald (1920)Wilhelm Ostwald (1920) Erwin Schrödinger (1925)Erwin Schrödinger (1925) Dorothea Jameson and Leo M. Hurvich (1955)Dorothea Jameson and Leo M. Hurvich (1955) Thorsten Wiesel, David Hubel, Russell Thorsten Wiesel, David Hubel, Russell

deValois (1960)deValois (1960) Edwin Land (1960-1985)Edwin Land (1960-1985)

23 problems

in colour vision

How did colour vision evolve?What is the neural code for colour?

How is colour organized in the cortex?How are colour differences scaled?

How to explain the aesthetics of colour?How does the eye adapt to light and colour?

Which are the cone inputs to cone-opponent cells?What is the significance of colour?

How do colour qualia come into being?How do colour opponent neurons interact?

How many colour vision deficiencies are there?How are cone-opponent cells spatially organized?

What is the status of elementary (opponent) colours?Which are the most important non-linearity in colour vision?

What are the neural correlates for white and black?What is the genetic basis of colour vision deficiencies?

How can colour constancy and adaptation be explained?How does neural activity correlate with colour perception?

How many (shifted) visual pigments can a photoreceptor have?How many dimensions do colour perception have (modes of appearance)?How is the relation between receptor specificity and pigments established?

How do ganglion cells pick receptors for receptive field centre and surround?Is a receptor identity (as L or M type) established solely by its pigment (or cant here

be an L type

with an M pigment?)

Area V1:

Eye dominance and orientation sensitive columns

The knots (corresponding to the circles in fig.A) are colour sensitive

Perception of light and colour.Model stages and important parameters.

Stimulus → sensory response → neural representation → perception

Stimulus: Spectral distribution of interference filters, Intensity, Size, Spatial contrast. 5 log units of intensity

Sensory response: Receptors (L,M,S), Light absorption and non-

linearity, Standing potentials, Adaptation

Neural representations: Firing frequency, Receptive field, Increment- and Decrement opponent cells (ON & OFF), Adaptation (retina and LGN)

Perceptual properties: Elementary colours (Y, R, B, G)Hue, Colour strength, Relative lightness (black, grey, white), Brightness.Object colours and light colours (reflecting and emitting sources of colour), Colour scaling

””Sense perception is the beginning of all research, Sense perception is the beginning of all research, and the truth of theoretical thought is given and the truth of theoretical thought is given exclusively by its relation to the sum total of exclusively by its relation to the sum total of those experiences.”those experiences.”

Albert Einstein, 1950Albert Einstein, 1950

””Nothing can exist in conscioussness without Nothing can exist in conscioussness without having first passed the senses.”having first passed the senses.”

AristotelesAristoteles