A Projection Method to Generate Anaglyph Stereoscopic Images

Eric DuboisVIVA Lab (Video, Image, Vision & Audio Research Lab)

School of Information Technology and Engineering (SITE)

University of Ottawa, Ottawa ON Canada

May 8, 2001 ICASSP 2001 2

Today’s Program

• Generalities on stereoscopic viewing• What is anaglyph?• Mathematical formulation of color stereo viewing• Mathematical formulation of anaglyph viewing• Method to generate anaglyphs by projection• Some results• What’s left to do?

May 8, 2001 ICASSP 2001 3

Stereoscopic Imaging

Scene Stereoscopiccamera

Stereoscopicdisplay

Viewer(with glasses)

May 8, 2001 ICASSP 2001 4

Stereoscopic Pair (Ray Hannisian)

May 8, 2001 ICASSP 2001 5

Stereoscopic display methods

• Stereoscope (1832)• Anaglyph (1853)• Polarized glasses (1891)• Liquid crystal shutter (time-sequential)• Autostereoscopic (no glasses)

May 8, 2001 ICASSP 2001 6

What is ‘anaglyph’?

Anaglyph is a method to view stereoscopic images using colored spectacles. The method was patented in 1891 by Louis Ducos du Hauron, but similar methods had been demonstrated previously by W. Rollmann in 1853 and J.C. D’Almeida in 1858.

May 8, 2001 ICASSP 2001 7

Origin of the word anaglyph

The word anaglyph is from the Greek

== again

== sculpture

May 8, 2001 ICASSP 2001 8

Classical method

For monochrome stereo images, the left view in blue (or green) is superimposed on the same image plane with the right view in red. When viewed through spectacles of corresponding colors but reversed, the three-dimensional effect is perceived.

May 8, 2001 ICASSP 2001 9

3D drawing

May 8, 2001 ICASSP 2001 10

Bubbles (André-Claude Latulippe)

May 8, 2001 ICASSP 2001 11

Ideal Stereoscopic Display

• Input left and right views:

• The three components j=1,2,3 are gamma-corrected RGB (in that order) that can be directly displayed by a standard RGB monitor.

• L is the sampling raster for the image

L xxx ;3,2,1),(),( '' jVV rjlj

May 8, 2001 ICASSP 2001 12

Ideal Stereoscopic Display• The three components go through the display gamma,

denoted g, and excite the RGB phosphors of the display,

• The spectral density functions of the RGB display phosphors are denoted

• The light emanating from point x in left and right images is given by

3,2,1),( jd j

jj

rjr

jj

ljl

dVQ

dVQ

3

1

3

1

,

,

xx

xx

3,2,1;,,' jrlVgV jj xx

May 8, 2001 ICASSP 2001 13

May 8, 2001 ICASSP 2001 14

Perceived stereo pair

• The color perceived at point x in the left and right images is determined by the projection of Ql(,x) and Qr(,x) onto the visual subspace using color matching functions kp

3,2,1,

,~

3

1

3

1

kVc

ddpV

dpQV

ljj

kj

jjklj

kllk

x

x

xx

May 8, 2001 ICASSP 2001 15

Perceived stereo pair

• In matrix notation,

• Similarly,• With the standard XYZ color space,

xCVxV ll ~

ddpc jkkjkj C

xCVxV rr ~

9109.01421.00357.0

0811.06592.02597.0

1808.03055.04641.0

C

May 8, 2001 ICASSP 2001 16

Perceived Stereo Image

• The value of the stereo image at each point x can be considered to be an element of a six-dimensional vector space S6

x

x

x

x

x

x

xV

3

2

1

3

2

1

~

~

~

~

~

~

~

r

r

r

l

l

l

V

V

V

V

V

V

May 8, 2001 ICASSP 2001 17

Visualization of an Anaglyph Image

• The anaglyph image is denoted L xx ;3,2,1,' jVaj

May 8, 2001 ICASSP 2001 18

Anaglyph Image

May 8, 2001 ICASSP 2001 19

Visualization of an Anaglyph Image

• The anaglyph image is denoted

• The light emitted from the screen at point x is

• where

L xx ;3,2,1,' jVaj

jj

aja dVQ

3

1, xx

xx 'ajaj VgV

May 8, 2001 ICASSP 2001 20

Perceived Anaglyph Image with Glasses

• The light from the CRT passes through two filters with spectral absorption functions fl() and fr() before arriving at the left and right eyes.

• The light spectral density at the left and right eyes is Qa(,x)fl() and Qa(,x)fr() respectively.

May 8, 2001 ICASSP 2001 21

Transmission of red and blue filters

May 8, 2001 ICASSP 2001 22

Perceived Anaglyph Image with Glasses

• The tristimulus values for the left and right views through the glasses are

• or Similarly,

3,2,1,

,~

3

1

3

1

kVa

dfdpV

dpfxQU

jajlkj

ljkj

aj

kllk

x

x

x

xVAxU all ~ xVAxU arr ~

May 8, 2001 ICASSP 2001 23

Goal of Anaglyph reproduction

We want the anaglyph image to appear as similar as possible to the ideal stereoscopic image

x

x

x

x

x

x

xV

3

2

1

3

2

1

~

~

~

~

~

~

~

r

r

r

l

l

l

V

V

V

V

V

V

x

x

x

x

x

x

xRVxU

3

2

1

3

2

1

~

~

~

~

~

~

~

r

r

r

l

l

l

a

U

U

U

U

U

U

Anaglyph3D

Ideal6D

May 8, 2001 ICASSP 2001 24

Generation of Anaglyph Image

• For each pixel location, we seek an element of the three-dimensional anaglyph subspace as close as possible to the original stereo image value which is in a six-dimensional space.

• If we have a suitable distance metric, we can choose the value in the three-dimensional space that is closest to the original six-dimensional value.

• This achieved by Projection

May 8, 2001 ICASSP 2001 25

Assumptions

• The approximation is carried out independently at each sample location

• The error metric at each point is a weighted squared error between and

• A global scaling of the Vaj is used to account for the attenuation of the filters.

U~ V~

VUWVUVU ~~~~~~ T2

May 8, 2001 ICASSP 2001 26

Projection

• Applying standard projection theory gives

• The 3 x 6 matrix is fixed and can be precomputed

xVWCRWRR

xVWRWRRxV

2T1T

T1T ~ˆ

a

2T1T WCRWRR

May 8, 2001 ICASSP 2001 27

Basketball Image

May 8, 2001 ICASSP 2001 28

Anaglyph

May 8, 2001 ICASSP 2001 29

May 8, 2001 ICASSP 2001 30

May 8, 2001 ICASSP 2001 31

May 8, 2001 ICASSP 2001 32

May 8, 2001 ICASSP 2001 33

May 8, 2001 ICASSP 2001 34

Conclusion

• A method to generate anaglyphs based on the mathematical description of the image display and perception process has been developed.

• Some color reproduction is possible.• Better stereo rendition than conventional

approaches.• You can view results at

http://www.site.uottawa.ca/~edubois/icassp01/

May 8, 2001 ICASSP 2001 35

What Else?

• How to choose filters to give best results?• Can color rendition be improved?• Can the method be applied to time-shuttered

stereo?

May 8, 2001 ICASSP 2001 36

Thanks for coming!