Copyright 2008 Society of Photo-Optical Instrumentation Engineers This paper was (will be) published in Conference Proceedings Volume 6803 Stereoscopic Displays and Applications XIX and is made available as an electronic reprint with permission of SPIE. One print or electronic copy may be made for personal use only. Systematic or multiple reproduction, distribution to multiple locations via electronic or other means, duplication of any material in this paper for a fee or for commercial purposes, or modification of the content of the paper are prohibited.

Large holographic displays as an alternative to stereoscopic displays

R. Häussler, A. Schwerdtner, N. Leister SeeReal Technologies GmbH, Blasewitzer Str. 43, 01307 Dresden, Germany

ABSTRACT

3D displays comprise stereoscopic displays and holographic displays. Eye convergence and accommodation are important depth cues for human vision. Stereoscopic displays provide only convergence information whereas holographic displays also provide accommodation information. Due to the inherently better 3D quality we consider holographic displays as the preferred alternative to stereoscopic displays. Our new approach to holographic displays omits unnecessary wavefront information and significantly reduces the requirements on the resolution of the spatial light modulator and the computation effort compared to conventional holographic displays. We verified our concept with holographic display prototypes and measurements. SeeReal’s approach makes holographic displays feasible as a consumer product for mass-market applications.

Keywords: 3D display, holographic display, stereoscopic display, eye convergence, eye accommodation

1. INTRODUCTION

Stereoscopic displays (which include autostereoscopic displays) and holographic displays are two approaches to display 3D information. Stereoscopic displays have been on the market for several years. They have reached a good image quality, e.g. regarding resolution and color reproduction, but haven’t yet succeeded in a breakthrough to a mass-market product. Holographic displays on the other hand are still on the laboratory or prototype level.

The principle of stereoscopic and holographic displays is fundamentally different. Stereoscopic displays provide different images of a 3D object for the left and for the right eye of an observer. These different images are generated either simultaneously or sequentially, i.e. either by spatial or by temporal multiplexing. Each image is displayed on a spatial light modulator (SLM) which in most cases is a liquid-crystal display (LCD). Each eye directly perceives the image as it is displayed on the SLM. Spatial or temporal beam-splitting techniques effect that each eye sees only its associated image.

Holographic displays are based on diffraction of light at a SLM. A hologram that is essentially a Fourier transform of the 3D object is encoded on the SLM. In the ideal case, light that is diffracted at the SLM with the encoded hologram has the same wavefront as the light that would be generated by a real existing object. Ideally, a holographic object reconstruction perfectly mimics a real object. An observer eye does not perceive the SLM itself but the 3D object that is reconstructed in space by the hologram.

In this article we will explain the limitations of stereoscopic displays, the conventional approach to holographic displays and the new approach by SeeReal Technologies. This will be followed by a description of our display prototypes and a presentation of measurement results. We will show that our approach has the potential to bring holographic displays to a mass-market consumer product.

2. STEREOSCOPIC AND HOLOGRAPHIC DISPLAYS

2.1 Stereoscopic displays and its limitations

There are two depth cues that characterize the difference in spatial vision on a stereoscopic display and on a holographic display: convergence of the eyes and accommodation, i.e. focus of the eye lens.

Fig. 1 demonstrates these depth cues. For normal viewing, an object (blue cube) is seen by both eyes. The eyes converge towards the object with a convergence angle � . The human vision system merges the two images seen by the eyes and deduces a depth information. The convergence is one depth cue. The other depth cue is accommodation. The eye lens

will focus on the object and thereby optimize the perceived contrast. Both depth cues provide the same depth information. The situation of normal viewing also applies to holographic displays as they mimic a real existing object by reconstructing the light wavefront that would be generated by a real existing object.

Stereoscopic displays inherently fail to provide a conclusive depth information. An autostereogram or a stereoscopic display provides two images with different perspective views, as indicated by the two small blue cubes at the display plane. The eyes will have the correct convergence angle � as for the real object (large blue cube in the background). However, the eye lenses will focus on the display plane as both images are displayed on the SLM in the display plane. Therefore, there is a mismatch between the depth information from convergence and accomodation.

Fig. 1. Comparison of eye convergence with convergence angle � and focus for normal viewing and an autostereogram. Normal viewing and also holographic displays provide the correct convergence and focus to an object (blue cube). An autostereogram only provides correct convergence whereas the focus is fixed to the display plane.

Furthermore, stereoscopic displays are only an approximation to natural viewing, as each image seen by the eyes is a flat image. All depth layers of an object that is extended in space appear in the same depth plane on the display.

The mismatch between convergence and focus may lead to vision problems like eye strain and fatigue if the mismatch is too large and if the stereoscopic display is watched for a long time. Recent theoretical and experimental investigations suggest that the depth range of a stereoscopic display should be limited to 13% of the observer distance1. This leads to a very small depth range, e.g. a depth range of 26 cm for a stereoscopic display used as a TV at typically 2 m observer distance.

This inherent mismatch between convergence and focus is therefore a severe limitation for the usability of stereoscopic displays.

2.2 Conventional holographic displays

Holographic displays provide the correct convergence and focus. Hitherto, their widespread usability was hampered by the required large number of pixels of the SLM.

The conventional approach to holographic displays reconstructs an object around the Fourier plane of the SLM. The size of the object reconstruction is limited to one diffraction order of the SLM as otherwise an overlap of multiple diffraction orders would be visible. The size h of one diffraction order is given by d/p

�h ⋅= , with the wavelength � , the distance d

between SLM and object and the pixel pitch p.

As an example, with � = 633 nm, d = 500 mm and p = 10 µm we get h = 32 mm. A small pixel pitch of 10 µm is needed for such a small object reconstruction. The absolute number of pixels depends on the viewing angle, i.e. the angle from which the object can be seen. Prototype systems use 15 million or 100 million pixels2,3 or a high-frequency acousto-optic modulator4. Estimations result in a pixel count of 1012 pixels for a full-parallax display with 0.5 m lateral object size and ±30° viewing angle3.

It is not only difficult to provide a SLM with such a high pixel count. A further difficulty is providing the data in real time. Calculating holograms with 1012 sampling points at a frame rate of 25 Hz would require an immense calculation power.

These numbers demonstrate that the conventional approach to holographic displays is not suitable for a mass-market consumer product.

3. SEEREAL’S APPROACH TO HOLOGRAPHIC DISPLAYS

3.1 Primary goal of the reconstruction

The fundamental difference between conventional holographic displays and our approach is in the primary goal of the holographic reconstruction. In conventional displays, the primary goal is to reconstruct the object. This object can be seen from a viewing region that is larger than the eye separation.

In contrast thereto, in our approach the primary goal is to reconstruct the wavefront that would be generated by a real existing object at the eye positions, creating virtual viewing windows at the 3D object. The reconstructed object can be seen if the observer eyes are positioned in or close to at least one virtual viewing window (VW). A VW is the Fourier transform of the hologram and is located in the Fourier plane of the hologram. The size of the VW is limited to one diffraction order of the Fourier transform of the hologram.

Fig. 2. Side view of a holographic display with lens, SLM, one object point and observer eye. The drawing shows the wavefront information that is generated in the conventional approach (red) and the essential wavefront information (green) that is actually needed at a virtual viewing window (VW). The essential wavefront information is encoded in a sub-hologram (SH) on the SLM. The dashed blue lines indicate a frustum in which the object points can be located.

Fig. 2 illustrates our approach and shows a Fourier transforming lens, a SLM and an eye of an observer. Coherent light transmitted by the lens illuminates the SLM. The SLM is encoded with a hologram that reconstructs an object point of a 3D object. An object with only one object point and its associated spherical wavefront is shown. It is evident that more complex objects with many object points are possible by superposing the individual holograms.

The conventional approach to holographic displays generates the wavefront that is drawn in red. The wavefront information of the object point is encoded on the whole SLM. The modulated light reconstructs the object point which is visible from a region that is much larger than the eye pupil. As the eye perceives only the wavefront information that is

transmitted by the eye pupil, most of the information is wasted. As an example, a holographic display for TV application with an observer distance of 2 m and a viewing angle of ±30° requires the wavefront information to be present in a zone of 2 m width. Only a small fraction of this zone is occupied by the eye pupils of an observer with an aperture of ca. 5 mm each. Most of the wavefront information is not seen and therefore wasted. In other words: much effort is done to project light into regions where no observer eye is.

In contrast thereto, our approach limits the wavefront information to the essential information. The correct wavefront is provided only at the positions where it is actually needed, i.e. at the eye pupils.

Fig. 2 shows a virtual VW which is positioned close to an eye pupil. The wavefront information is encoded only in a limited area on the SLM, the so-called sub-hologram (SH). The position and size of the SH is determined geometrically by projecting the VW through the object point onto the SLM. This is indicated by the green lines from the edges of the VW through the object point to the edges of the SH. Only the light emitted in the SH will reach the VW and is therefore relevant for the eye. Light emitted outside the SH and encoded with the wavefront information of the object point would not reach the VW and would therefore be wasted. This is indicated in Fig. 2 by the green spherical wavefronts for the essential information and the red spherical wavefronts for the wasted information. The observer will not notice that the wavefront information outside the VW is not present or not useful as long as each eye pupil is in a VW.

The VW has to be at least as large as the eye pupil and at most as large as a diffraction order in the observer plane. This ensures that light from only one diffraction order will reach the VW. Light emanating from other diffraction orders of the reconstructed object point is outside the VW and is therefore not seen by the eye.

The size of the SH depends on the distance of the point from the SLM. The size of the SH is the same as the size of the VW for a point halfway between SLM and VW. The VW has a typical size of the order of 10 mm.

The essential idea of our approach is that for a holographic display the highest priority is to reconstruct the wavefront at the eye position that would be generated by a real existing object and not the to reconstruct the object itself.

3.2 Required pitch of the SLM

For conventional holographic displays, the reconstructed object is essentially the Fourier transform of the hologram. The diffraction angle of the SLM determines the size of the reconstructed object and hence a small pixel pitch is needed. A large object reconstruction (e.g. with 20 diagonal) would require a pixel pitch of the order of 1 µm.

The requirements on the pitch of the SLM are significantly lessened by our approach. The VW is the Fourier transform of the hologram. The diffraction angle of the SLM determines the size of the VW. A moderate pixel pitch of 50 µm generates a VW with lateral size of 20 mm at a distance of 2 m. These are typical values for a holographic display for TV applications. The size of the reconstructed object is not limited by the pixel pitch but by the size of the SLM. The 3D object can be located anywhere in a frustum defined by the VW and the SLM. This frustum is indicated by the dashed blue lines in Fig. 2. The 3D object can be located in front of and behind the SLM.

Therefore, our approach makes holographic displays for large object reconstructions feasible.

3.3 Hologram calculation

There are two methods to calculate the hologram5.

The first method is evident from Fig. 2. The hologram is calculated from the object point-by-point. The SH of each object point is calculated and appropriately sized and positioned. The final hologram is generated by superposing the SHs of all object points.

The second method comprises three steps.

• In a first step the object is sliced in layers parallel to the SLM and each layer is transformed to the VW by a Fresnel transform. The Fresnel transform differs from a Fourier transform by spherical phase factor. This is repeated for all object layers.

• In a second step the Fresnel transforms calculated in the first step are summed up to a superimposed wavefront in the VW. This superimposed wavefront represents the wavefront that would be generated by real existing object.

• In a third step the superimposed wavefront calculated in the second step is transformed to the SLM by a Fourier transform. The result of the Fourier transform is the hologram that reconstructs the wavefront that would be generated by a real existing object in the VW.

The two methods differ in that the first method treats the object point-by-point whereas the second method treats it layer-by-layer. The number of calculations of the first method is larger as there are more object points than object layers. This is counterbalanced by the fact that the SHs of the first method are smaller than the layers of the second method. The first method can be efficiently executed on a graphics card in real time, i.e. with 25 frames per second for an object in HDTV resolution.

3.4 Observer tracking

The display is equipped with an eye position detector and tracking means. Hence it is possible to reduce the size of a VW to the size of approximately an eye pupil. Two VWs, i.e. one for the left eye and one for the right eye, are always located at the positions of the observer eyes. The two VWs may be generated by temporal or spatial multiplexing. Additional VWs for several observers may be generated in the same way. Thus the viewing angle of the reconstructed object can be enlarged without increasing the resolution of the SLM.

The eye position detector and tracking means always locate the VWs at the observer eyes. There are two alternatives for the tracking means.

• Light source tracking Shifting the position of the light source also shifts the position of the VW. The position of the light source does not have to be shifted mechanically. A light source may be an activated pixel in an additional LCD that is illuminated by a homogenous backlight. By activating a pixel at the desired position on the LCD the light source can be shifted electronically without mechanical movement.

• Beam-steering element With a beam-steering element after the SLM the optical path from the light source to the SLM can be kept constant. This is advantageous with respect to light efficiency and lens aberrations. The beam-steering element deflects the light after the SLM and directs the light towards the observer eyes. As an example, the beam-steering element may comprise prism-shaped cells that are filled with liquid crystals. The effective refractive index and hence the deflection angle is controlled by a voltage applied to electrodes at the cells.

Additionally, the locations of the SHs on the SLM are also adapted to the positions of the observer eyes.

3.5 Visual resolution of the object

In our approach the wavefront information of an object point is encoded in a small area only, the so-called SH. This area limits the aperture of the light that reconstructs an object point. In principle, the resolution of the reconstructed object point is limited by diffraction at this aperture. This limitation would be visible on a screen positioned at the object plane. This is similar to diffraction-limited imaging with lenses where the resolution of the image is limited by diffraction at the lens aperture.

However, limiting the encoded wavefront to a SH does not affect the visual resolution with which an observer sees the reconstructed object. The observer eye is part of the optical system of holographic display and observer eye. The limiting aperture in this optical system is the eye pupil as the area of the SH corresponds to the area of the VW and as the VW is larger than the eye pupil. The visual resolution of the object is therefore determined by the resolution of the eye and not by the resolution of the holographic display. Limiting the encoded wavefront to a SH omits therefore details of the object which cannot be seen anyway.

4. PROTOTYPES

We verified our approach with two prototypes. A monochrome 20 inch prototype uses a large SLM with 20 inch diagonal. A full-color 8 inch prototype comprises a small SLM that is optically enlarged by magnification optics. Both prototypes are combined with real-time computation of the holograms on a high-end graphics card of the nVidia GeForce 8000 series with 25 frames per second.

4.1 Setup and description of the 20 inch prototype

Fig. 3 shows the setup of the 20 inch prototype. The components are LED backlight, shutter display, Fourier lenticular, SLM and beam-splitting lenticular (from left to right).

Fig. 3. Setup of the 20 inch prototype. The components are (from left to right): LED backlight, shutter display, Fourier lenticular, SLM and beam-splitting lenticular. The inset shows two interlaced holograms on the SLM that are separated by the beam-splitting lenticular in order to generate two VWs.

The LED backlight consists of 144 high-brightness LEDs with wavelength � = 630 nm. Each LED has a parabolic reflector and small emission angle. The spectral linewidth (FWHM) � � = 20 nm is sufficiently narrow to provide sufficient temporal coherence.

The SLM used in this prototype is a standard monochrome LCD with 20 inch diagonal and 9 million amplitude-modulating pixels. The pixel pitch is 207 µm horizontal and 69 µm vertical. The hologram was encoded in the display by detour-phase encoding using three pixels to represent one complex number6. The size w of the VW is one third of one diffraction order and is given by d/p

�w ⋅= .

This results in a VW size w = 6 mm with a wavelength � = 630 nm, an observer distance d = 2 m and a pitch p = 207 µm that is needed to encode one complex number. The hologram is a vertical-parallax-only hologram with holographic reconstruction in the vertical direction. The vertical VW size w = 6 mm is sufficient for an eye pupil and facilitates eye focusing to a reconstructed object point.

A single Fourier-transforming lens with 20 inch diagonal that images the light source into the observer plane would be too bulky. Instead, a lenticular comprising approximately 60 horizontal cylindrical lenses is used. Each cylindrical lens is illuminated by a horizontal line light source. A line light source is comprised of activated pixels in an additional LCD which serves as a shutter display and which is illuminated by the LED backlight. The width of the line light source is sufficiently narrow to provide sufficient spatial coherence. These line light sources and the lenticular are aligned such that all images of the light sources coincide in the observer plane. The display with the encoded hologram is illuminated by the multitude of light sources, and each lens performs an optical Fourier transform of a part of the hologram in the vertical direction. As the light sources are mutually incoherent the eyes see a reconstructed object that is composed of mutually incoherent partial objects. This may lead to brightness inhomogeneities that can be compensated in the hologram calculation.

Two VWs, one for each eye, with a horizontal separation of approximately 65 mm are generated by spatial multiplexing. Two holograms are interlaced in the display. A lenticular is used as a beam-splitting element and projects light from one hologram to the left eye and light from the other hologram to the right eye. This technique is well known from stereoscopic displays. In our case, each eye sees a holographic object reconstruction that is generated by its own hologram.

An eye position detector controls the positions of the VWs. Vertical and axial tracking is achieved by adapting the positions of the line light sources on the shutter display. Horizontal shifting of the hologram content on the SLM facilitates horizontal tracking.

Thus we achieve a holographic object reconstruction with 20 inch diagonal and beyond for a moving observer. A VW is located at the position of each eye, and each eye sees its own holographic reconstruction. The left-eye hologram and the right-eye hologram are calculated from its respective perspective views of the object. This combination of holographic reconstruction and stereoscopy avoids the mismatch between eye focusing and convergence that is inherent in stereoscopic displays. Thus, the observer is provided with true depth information of a holographic object reconstruction.

4.2 Description and setup of the 8 inch full-color prototype

Fig. 4 shows the schematic setup of the 8 inch holographic projection display7. This prototype uses a small SLM, i.e. a micro display that is usually used in projectors. The SLM has HDTV resolution with 1920 * 1080 pixels and a pixel pitch of 8 µm. A micro display has the advantage that it can be optically enlarged with magnification optics to the desired size. Furthermore, phase-modulating SLMs are hitherto only available as micro displays. A phase-modulating SLM is advantageous for a holographic display as the diffraction efficiency is higher than for an amplitude-modulating SLM.

Fig. 4. Schematic setup of the 8 inch holographic projection display. It comprises a light source (LS), a SLM, several lenses, a filter and a large concave mirror.

The light source (LS) illuminates the SLM in plane Z0 via a beam expander. The light source comprises three lasers at wavelengths 475 nm, 532 nm and 635 nm that are combined with dichroic mirrors. The lasers are switched synchronously with the SLM to provide full-color holograms sequentially.

The lens in plane Z1 images the SLM onto a concave mirror with 8 inch diagonal in plane Z3. The SLM with a diagonal of 0.7 inch is thereby optically enlarged to an image with 8 inch diagonal. The Fourier transform of the SLM is located in plane Z2. A filter in plane Z2 transmits only the 1st diffraction order and blocks other diffraction orders. The concave mirror images the aperture of the filter into the observer plane Z4. The imaged aperture is the VW through which the observer sees the reconstructed 3D object.

The 3D object can be anywhere in the frustum defined by the VW and the imaged SLM in Z3. Its lateral size is up to 8 inch for objects between the VW and Z3 and is even larger for objects behind Z3. The observer distance between the concave mirror in Z3 and the VW in Z4 is 1.3 m. The size of the VW is determined by the magnified pitch of the imaged SLM in Z3.

The holographic projection display comprises two setups shown in Fig. 4, one for each eye. These setups share the same concave mirror. One setup generates the VW for the left eye whereas the other setup generates the VW for the right eye.

4.3 Measurements

Fig. 5 demonstrates the reconstruction of a 3D object that is extended in depth. The photographs were taken at the 8 inch holographic projection display. The circles are actually in different colors, however the camera is monochrome only.

Fig. 5. Two photographs of circles reconstructed with the 8 inch holographic projection display. The camera lens was positioned in the VW with the lens aperture being smaller than the VW. The circles are at different depths. The camera lens was focused on the large circles at 50 cm in front of the display (upper photograph) and on the small circles at 50 cm behind the display (lower photograph).

The photographs in Fig. 5 were taken with the camera positioned in the observer plane and the lens aperture in one VW. The lens aperture was smaller than the VW. The camera therefore mimics an observer eye that is located in the VW. The 3D object consists of circles at different depths. For the upper photograph in Fig. 5 the camera lens was focused on the three large circles which are reconstructed 50 cm in front of the display. The two small circles which are reconstructed 50 cm behind the display are not in focus. For the lower photograph the camera lens was focused on the two small circles 50 cm behind the display. Now the three large circles are not in focus.

Fig. 5 demonstrates the holographic nature of our display. It is evident that the circles are reconstructed at different depths. Only the circles on which the focus of the camera lens is set are in focus whereas the circles at other depths are not in focus. The same also holds for an observer eye: an observer perceives parts of a 3D object that are in focus whereas other parts of the 3D object are not in focus. The holographic display therefore mimics natural viewing.

The depth cue of focus information can only be provided by a holographic display. A stereoscopic display cannot reconstruct parts of a 3D object at different depths and does not provide the correct focus information for the eyes.

5. CONCLUSIONS AND OUTLOOK

The verification of our approach with two prototypes shows that large holographic displays are possible with today’s technology. We demonstrated a monochrome holographic display with 20 inch diagonal and a full-color holographic projection display with 8 inch diagonal. Both prototypes are combined with real-time calculation of the holograms on a graphics card of a PC.

The highest priority of holographic displays of other research groups2,3,4 is to reconstruct the 3D object itself. These displays need a high-resolution SLM and generate small reconstructed objects with a lateral size of the order of 10 cm. SeeReal’s approach is based on the idea that the observer eye is part of the optical system. The highest priority is to reconstruct the wavefront that would be generated by a real existing object at the observer eyes. If the holographic display is combined with an eye position detector and tracking means it is sufficient to provide this wavefront in a small area with a size of approximately the eye pupil. Our approach allows to use a SLM with moderate resolution and significantly reduces the computation effort of the hologram calculation.

SeeReal’s approach facilitates large holographic displays for mass-market applications, e.g. holographic TV and gaming. Our prototypes were built using off-the-shelf components. A consumer product requires components that are optimized for holographic displays. These modifications are possible with today’s technology. As an example, the alignment of the LC molecules and the thickness of the LC layer in a LCD have to be adapted in order to provide a phase-modulating SLM instead of an amplitude-modulating SLM.

We think that stereoscopic and holographic displays can co-exist in the market of 3D displays. There are different market segments for stereoscopic and holographic displays.

• Stereoscopic displays suffer from the inherent mismatch between eye convergence and focus. This may lead to eye fatigue and significantly limits the depth range of 3D objects. Stereoscopic displays can be used for applications where the risk of inappropriate use is limited. Examples are mobile displays for short-term use (e.g. 3D cell phones), cinemas with large viewing distances and high-end professional displays where the user controls the depth range (e.g. for CAD engineering).

• Holographic displays mimic natural viewing and provide the correct depth cues of convergence and focus. They are therefore suitable for long-term use and large natural depth ranges. Examples are consumer displays for TV, PC and gaming applications and also high-end professional displays.

In conclusion, our approach to holography is a significant step toward practical application of holographic displays. We consider holographic displays as the preferred alternative to stereoscopic displays for a consumer product in the market of 3D displays.

REFERENCES

1 H. Choi, J. Kim, J. B. Park and B. Lee, "Analysis on the optimized depth of 3D displays without an accommodation error", IMID 07 Digest, 1811 – 1814 (2007). 2 K. Maeno, N. Fukaya, O. Nishikawa, K. Sato and T. Honda, "Electro-holographic Display using 15 Mega Pixels LCD", Proc. SPIE Vol. 2652 Practical Holography X, 15 – 23 (1996). 3 C. Slinger, C. Cameron, S. Coomber, R. Miller, D. Payne, A. Smith, M. Smith, M. Stanley and P. Watson, "Recent developments in computer-generated holography: toward a practical electroholography system for interactive 3D visualisation", Proc. SPIE Vol. 5290 Practical Holography XVIII, 27 – 41 (2004). 4 P. St.-Hilaire, S. A. Benton, M. Lucente, J. D. Sutter and W. J. Plesniak, "Advances in Holographic Video", Proc. SPIE Vol. 1914 Practical Holography VII, 188 – 196 (1993). 5 A. Schwerdtner, N. Leister and R. Häussler, "A New Approach to Electro-Holography for TV and Projection Displays", SID 07 Digest, 1224 – 1227 (2007). 6 C. B. Burckhardt, "A Simplification of Lee’s Method of Generating Holograms by Computer", Applied Optics 9(8), 1949 (1970). 7 N. Leister, A. Schwerdtner, G. Fütterer, S. Buschbeck, J.-C. Olaya and S. Flon, "Full-color interactive holographic projection system for large 3D scene reconstruction", to be published in Proc. SPIE Vol. 6912 (2008).