two kinds of novel multi-user immersive display systems
TRANSCRIPT
Two Kinds of Novel Multi-user Immersive Display Systems
Dongdong Guan1,2 Chenglei Yang1* Weisi Sun1 Yuan Wei1 Wei Gai1,2
Yulong Bian1,2 Juan Liu1,2 Qianhui Sun1 Siwei Zhao1 Xiangxu Meng1,2
1Shandong University
Jinan, China
chl_yang, [email protected]
2Engineering Research Center of Digital Media
Technology, Ministry of Education
Jinan, China
ABSTRACT
Stereoscopic display is a standard display mode for virtual
reality environments. Typical 3D projection provides only a
single stereoscopic video stream; thus co-located users
cannot correctly perceive the virtual scene based on their
own position and view. Several works devoted to developing
multi-user stereoscopic display, but the number of users is
very limited or the technical implementation is complicated.
In this paper we put forward two flexible and simple
projection-based multi-user stereoscopic display systems.
The first one, named TPA, is based on a triple-projector array
and provides a 120Hz active stereo for three users. Two
TPAs can be combined to form a six-user system. The second
one, named DPA, is a dual-projector and easy-implemented
system providing individual stereoscopic video stream for
two to six users. Finally, a co-located multi-user virtual
fireman simulation training system and a virtual tennis
simulation system were created to verify the effectiveness of
our systems.
Author Keywords
Multi-users stereoscopic display; virtual reality; co-located
collaboration.
ACM Classification Keywords
H.5.1. Information interfaces and presentation: Multimedia
Information Systems—Artificial, augmented, and virtual
realities
INTRODUCTION Stereoscopic display is very popular in cinema and virtual
reality environments providing the audience with immersive
experience. This technology is implemented by displaying
two-channel videos of left and right eyes on the same screen
such that users can gain stereoscopic visions with active
stereos or polarizers.
At present, typical 3D televisions and 3D projectors provide
only a single view for all audiences; thus in co-located
collaborative virtual environments, they offer only a correct
perspective view for a single user, causing other co-located
users’ views more or less distorted. [12] analyzes the
influence caused by viewpoint distortion of group users in a
single-view CAVE environment; [10,15] endeavor to display
average view image to compensate for the distortion, but fail
to offer a correct perspective view for each user. On the other
hand, co-located users cannot roam in virtual environments
independently, but only follow a leader user. Thus multi-user
display is essential to improve reality experience and
flexibility in co-located collaborative virtual environments.
Compared with 3D television and Head Mount Display
(HMD), projection-based stereoscopic display can support
large-scale display and allow users move freely in physical
spaces. Multi-user projection can provide a correct
perspective view for each user and improve their sense of
reality experience and immersion. This technology has been
applied in fields such as virtual surgery [2], urban design [3],
and games [17].
Although most of the current projection-based multi-use
display systems can only serve two users [1,2,4,14], the
C1×6 System in [8] can serve six users with six projectors
working concurrently by modifying the projectors (removing
color wheels, adopting a fixed color filter, etc.). The C1×6
System owns the largest user capacity, but the technique
realization and the modification made to the projector (such
as removing color wheels and image calibration) are
complex and expensive, and the construction is not flexible
for two to five users. Furthermore, each adjusted projector in
a C1×6 System cannot be used as a color projector alone
anymore.
Therefore, in this paper, we put forward two projection-
based multi-user stereoscopic display systems, which can be
easily implemented. They are also flexible in building multi-
user collaborative VR systems and conducting related
human-computer interactive research. The first system is
named TPA (Triple-Projector Array), and the second system
is named DPA (Dual-Projector Array). TPA is based on a
triple-projector array, which provides a 120Hz active stereo
for three users, and two TPAs can be combined as a six-user
system. DPA is a dual-projector and easy-implemented
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies
bear this notice and the full citation on the first page. Copyrights for
components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to
post on servers or to redistribute to lists, requires prior specific permission
and/or a fee. Request permissions from [email protected]. CHI 2018, April 21–26, 2018, Montreal, QC, Canada
© 2018 Association for Computing Machinery.
ACM ISBN 978-1-4503-5620-6/18/04…$15.00
https://doi.org/10.1145/3173574.3174173
CHI 2018 Paper CHI 2018, April 21–26, 2018, Montréal, QC, Canada
Paper 599 Page 1
system providing individual stereoscopic video streams for
two to six users. In summary, this paper makes the following
contributions:
(1) We propose TPA, which is a projection-based multi-
user stereoscopic display system with three projectors.
TPA can implement stereoscopic projection for three users.
Compared with the C1×6 System, TPA only needs a simple
modification in the color wheel of the projector. It is easier
build while possessing lower cost and higher refresh rate at
the same time. Additionally, two groups of three-user
arrays can be extended to six users combined with the
optical polarization, resulting in more flexibility for users.
Furthermore, each adjusted projector in a TPA can still be
used as a color projector alone afterwards.
(2) We also present DPA, which is another projection-
based multi-user stereoscopic display system with two
projectors. DPA does not require modification to the
internal structure of the projector. With the help of
dedicated shutter glasses we designed in this paper, it can
flexibly provide individual stereoscopic video streams for
2 to 6 users. Its implementation is easier and its cost is
lower than those of the C1×6 System and TPA, so that it
can provide more convenient environment construction for
co-located collaborative virtual environments, although its
refresh rate may be reduced when the number of users
increases.
(3) One force-to-zero driving strategy is put forward to
design a kind of dedicated shutter glasses that can keep the
open/close sequence of the LC shutter in sync with the
projection displaying sequence, to achieve users’
independent viewing. It can reduce the switching time of
LCD, thus avoiding crosstalk at high refresh rates.
(4) Two application examples are developed for verifying
the effectiveness of our work proposed in this paper. One
is the multi-user virtual fireman simulation training system
based on DPA, and the other is the multi-user virtual tennis
simulation system based on TPA.
The rest of the paper is organized as follows: the first is a
brief review of related work; the second is a realization of
TPA; the third is a presentation of DPA; the fourth is a
demonstration of the two application examples; the fifth is
discussion and the final is conclusion.
RELATED WORK There are three main ways to achieve stereoscopic display:
lenticular, optical polarization, and active stereo, based on
which multi-user display systems could be developed. When
a multi-user stereoscopic display system serves n users, there
are totally 2n-way videos displaying on the same screen area
concurrently: n-way for left eyes and n-way for right eyes.
The key issue is to enable users to get their own image from
overlapping images on the same screen.
[7,11,13] employ lenticular or parallax barrier displays to
implement a two-viewpoint display system; [9] employs
LCD 3D screen combined with active stereo to provide
haptic interaction for two users. A drawback of these systems
lies in that users are limited to a relatively small area in these
LCD screen-based systems.
Several works are devoted to projection-based two-user
display systems. [1] proposes a system that employs a 144Hz
CRT Projector to realize active stereoscopic displays for two
users, which results in 36Hz per eye per user. Combining
optical polarization and two active 3D projectors, Barco et al.
[2] develops the “Virtual Surgery Table” for two users. This
is also applied to a large display wall by Riege et al. [14]. A
similar technology is used in [4] to design a CAVE system
in co-located collaborative interaction research for two users.
A common weakness of these systems is that they can serve
only two users.
The C1×6 System in [8] uses the active stereo projection in
combination with optical polarization to realize a
stereoscopic display function for six users. It employs 6
projectors to construct a projection array and display
individual stereo video streams for 6 users. With three
projectors for each of the left and right eyes, the
corresponding left or right video is displayed respectively,
which is distinguished by a polarizer. The popular single-
chip DLP projectors are adopted to construct a projection
array. The color wheel of the projector is removed to adjust
a 120Hz color projector to a 360Hz monochromic projector
and three projectors work in parallel to improve the standard
120Hz image refresh rate to 360Hz; thus 120Hz image
refresh rate is ensured for each user. Moreover, dedicated
shutter glasses are designed and toggled in sync with the
display sequence; therefore each user can watch an exclusive
stereoscopic video stream.
System type Array
projector number
User capacity
Monocular refresh rate
of each user
Frequency of shutter
glasses
Stereoscopic combination
mode
Whether to modify projector structure
Whether to modify projector
circuit
Whether to modify
glasses circuit
Agrawala[1] 1 2 36Hz 144Hz N N Y
Riege[14] 2 2 60Hz 120Hz User N N N
C1×6
System[3] 6 6 60Hz 360Hz
Left and
right eyes Y Y Y
TPA 3/6 3/6 60Hz 360Hz User Y N Y
DPA 2 2-6 =<60 Hz 120Hz Left and
right eyes N N Y
Table 1. Comparison among different systems.
CHI 2018 Paper CHI 2018, April 21–26, 2018, Montréal, QC, Canada
Paper 599 Page 2
Figure 1. TPA: overview of triple-projector array system.
[3] develops a collaborative urban design system for six
users based on the C1×6 System. However, the removal of
the color wheel causes the temperature of the projector to rise;
thus extra cooling module is needed, and the engineering
complexity is increased. Moreover, the adjusted projectors
cannot be used as normal ones. The driven strategy of
dedicated shutter glasses is also complex.
Table 1 lists the performance and implementation of the
existing multi-user projection systems and our proposed
systems.
TPA: TRIPLE-PROJECTOR ARRAY SYSTEM
TPA in this research uses three commercial single-chip DLP
active 3D projectors to construct a projection array, allowing
120Hz active stereoscopic for three users. Dedicated shutter
glasses are also designed to ensure that each user can watch
their own videos. Figure 1 shows the construction and
operation of TPA.
Triple-projector Array
With the increase of the number of users, the image refresh
rate of the projector must be improved, and based on the
working mode of commercial single-chip DLP projectors,
three projectors can work in parallel to triple the rate. For a
single-chip DLP projector, the RGB bit plane are projected
respectively synchronizing with the rotating motion of the
color wheel. When the red section of the color wheel turns in
front of the lamp, the red bit plane is projected. The same is
true for the green and blue bit planes. The colors are thus
displayed sequentially at a sufficiently high rate that the
observer sees a composite “full color” image. From this we
can see that the projection process of one full color image
can be divided into three monochromatic intervals. Therefore,
one 120Hz color projector can be regarded as one 360Hz
monochromatic projector, and three 120Hz projectors can
work concurrently to realize the 360Hz full color refresh rate.
During the assembly of the three projectors, their color
wheels of the C1×6 System are removed with R-G-B filters
respectively in front of the projector lens instead, so that each
of the three projectors respectively is adjusted to a
monochromatic projector to display a primary bit plane of
the images. However, as the color wheel is removed, which
causes temperature rise of the projector, extra cooling
module is needed, so the complexity of engineering is
increased. In contrast, TPA changes the color sequence of the
color wheel, and through GPU-based image mixing, the full-
color images of the three users can be composited in each
monochromatic interval respectively. Since the color wheel
is not removed, there is no rise in temperature and so no more
cooling module is needed.
The modification of the color wheel is shown in Figure 2: the
first projector retains its original RGB sequence; the color
wheel of the second projector is modified to GBR sequence;
and the third modified to BRG sequence.
(a) (b) (c)
Figure 2. Modification of the sequence of color wheels. (a)
Original R-G-B sequence of projector 1;(b) G-B-R sequence of
projector 2;(c) B-R-G sequence of projector 3.
After replacing color wheel, the primary components of the
three projectors in each monochromatic interval is shown in
Figure 3c. In the first monochromatic interval, the three
projectors respectively display R-G-B components of User 1;
in the second interval, they respectively display G-B-R of
User 2; and in the third interval, they respectively display B-
R-G of User 3. Therefore, three images of the three users are
composited in order in each interval. Figure 3 compares the
composite mode of the color image on normal projectors, the
C1×6 System and TPA.
Real-time Image-mixing and Geometric Alignment Based on GPU
To fit the display mode of the triple-user projection array, the
three independent video images need to be mixed to form
triple-user stereoscopic video stream format. Different from
the classical mode, each projector displays a mixed image
consisting of three primary components extracted from three
video images respectively. Whatever the arrangement of the
CHI 2018 Paper CHI 2018, April 21–26, 2018, Montréal, QC, Canada
Paper 599 Page 3
projector array is, it only can reach a rough image registration,
so image deformation for three projectors is necessary to
achieve a precise per-pixel alignment. The processing flow
of image data by TPA, including GPU-based image mixing
and alignment processing is shown in Figure 4.
(a)
(b)
(c)
Figure 3. Comparison of the color image composite modes on
(a) normal projectors, (b) C1×6 system and (c) TPA.
Similar to the alignment processing in the C1×6 System, this
research also adopts camera-to-projector calibration
algorithm based on Gray code patterns [5]. After identifying
common projection on the screen, a look-up table is
generated for each projector based on the calibration results,
and deformation processing is conducted to realize the image
registration of three projectors. Because the mixing process
and the look-up tables are fixed, GPU acceleration can be
adopted to ensure real-time rendering and displaying.
TPA conducts real-time rendering of virtual scene with
Unity3D engine. It renders left and right eye image of three
users in each frame respectively at a 60FPS frame rate. Each
three left or right images are composited into one tripled-size
image, which is output by graphics card, after image mixing
and deformation processing. It is distributed averagely to
three projectors due to the screen splicing function of the
graphics card.
Dedicated Shutter Glasses
Dedicated shutter glasses should be designed to keep the
open/close sequence of the LC shutter in sync with the
projection displaying sequence, to achieve users’
independent viewing. Classical commercial shutter glasses
are designed for single view, with the shutter glasses toggling
per 1/120 second, so the user can perceive left-eye image and
right-eye image with corresponding eye. The interval of open
(white) state and close (black) state is equal as 1/120
second≈8.33ms. To match the display format of the triple-
projection array, the dedicated shutter glasses also toggle per
1/120 second, but the opening interval is reduced to 1/3 of
the previous, meaning that they only open in their own
monochromatic interval, and close in other two intervals to
avoid crosstalk. The open/close timing of classical shutter
glasses and our shutter glasses are shown in Figure 5.
Figure 4. The processing flow of image data by TPA.
CHI 2018 Paper CHI 2018, April 21–26, 2018, Montréal, QC, Canada
Paper 599 Page 4
Figure 5. Open/close timing of dedicated shutter glasses.
As the opening time is reduced to 1/360s≈2.78ms, LC
glasses need to open/close at a fast switch speed, which
regular LC shutters fail. Due to the property of liquid crystal
molecule, LC shutter has asymmetrical switch time, with
short closing time (from white to black) less than 0.1ms and
longer opening time (from black to white) of around 2.5ms.
During the opening period, LC transmittance gradually
increases in curved shape. This opening time, compared with
display period of 360Hz, is obviously too long. If LC shutter
opens in sync with the corresponding monochromic interval,
longer opening time result in insufficient brightness, and if
LC glasses open in advance, crosstalk may be caused. Thus
the important issue is to reduce LC opening time in designing
dedicated shutter glasses.
C1×6 uses two layers of differently configured regular LC
shutters, and reduces its opening time through asynchronous
open/close driving, while this method is complex to realize,
and the transmittance decreases due to shutter overlap. This
research puts forward an easier driving strategy which we
call force-to-zero driving, to reduce the opening time of LC
shutter.
Typically, LC shutter is driven by H-bridge circuit. H- bridge
has three-state output: +15V, 0V and -15V. When driving
voltage is 0V, LC is in open (white) state, when driving
voltage turns +15V or -15V, LC glasses is in close (black)
state, and when H-bridge changes from ±15V to 0V, LC
glasses need about 2.5ms to fully open (white). Similar to the
overdriving strategy of LCD screen in [6,16], when driving
voltage reduces from +15V to 0V, a short-time - 15V pull-
down interval is inserted to speed up its process back to 0V.
The effectiveness of force-to-zero strategy depends on the
duration of the pull-down interval. If the duration is
insufficient or too long, its opening time cannot be reduced,
and based on experiments, when the duration of pull-down
interval is 10-15μs, the opening time of LC glasses can be
reduced to about 0.2ms, being able to satisfy the display
speed requirement of 360Hz or higher. Equally, when
driving voltage changes from -15V to 0V, a pull-up interval
should also be added. Force-to-zero strategy can be
completed by embedded single chip machine in shutter
glasses, and it does not increase the circuit complexity
considerably. Figure 6 compares LC driving with the
overdriving zero crossing driving in this research.
(a)
(b)
Figure 6. Comparison between typical H-Bridge LC driving
and Force-to-zero driving strategy of LC glasses, (a) Typical
H-Bridge LC Driving, (b) Force-to-Zero LC Driving.
To realize the synchronized display sequence between
open/close timing of LCD shutter glasses and the projector
array, the build-in TI DLP Link mode inside the projector is
adopted. When the projector displays one frame of image, it
projects a high-brightness optical pulse of 20μs as frame
synchronization signal, which is received by shutter glasses
through photodiode. As the time of optical pulse is rather
short, human eyes cannot perceive it.
Triple-projector array and dedicated shutter glasses can offer
individual 120Hz active stereoscopic display for three users.
As the commercial 3D projector is used, the modification
process of projection array is much easier than C1×6, only
necessary modification in color wheel of the projector and
no modification in control circuit. The C1×6 System needs
CHI 2018 Paper CHI 2018, April 21–26, 2018, Montréal, QC, Canada
Paper 599 Page 5
projection array composed by six projectors to work
simultaneously, while in this research, three projectors as one
group may be used to realize active 3D projection for three
users, and two groups of three-user array can be extended to
six users combining with optical polarization, which is more
flexible to use.
DPA: DUAL-PROJECTOR ARRAY SYSTEM
DPA employs dual projector as a projection array to provide
correct perspective view for 2 to 6 users. It can provide
independent 120Hz 3D display for 2 users, and for more than
2 users, refresh rate of each user is scaled down, but it is easy
for system implementation, without modification to the
projector, and only dedicated 3D shutter glasses are needed.
Two projectors respectively display the images of users’ left
and right eyes, which is distinguished by optical polarizer.
The standard 120Hz projectors display the 3D video of each
user in order. When the user number is n, projection is
divided into n intervals, Projector 1 displays according to the
sequence of “User 1 left image – User 2 left image - … - User
n left image”, and Projector 2 displays according to the
sequence of “User 1 right image – User 2 right image - … -
User n right image”. In each interval, the screen
demonstrates the left image and right image of a certain user
at the same time, and the two LC shutters open in the
meantime to form stereoscopic vision. In other intervals, they
close at the same time to avoid crosstalk. When the user
number is more than 2, the single- eye refresh rate of each
user is scaled down, that is 40Hz for 3 users, 30Hz for 4 users,
24Hz for 5 users and 20Hz for 6 users.
Unity3D engine is employed to perform real-time rendering.
When the user number is n, n pairs of virtual cameras need
to be set, respectively corresponding to n left eyes and n right
eyes. To ensure the left-eye and right-eye image shown
simultaneously on the screen, the left and right camera
rendering result of each user needs to be set as a double-size
stereo image side by side, and the image is displayed by two
projectors due to the screen splicing function of the graphics
card.
The two projectors in DPA is divided based on left and right
eyes, whose advantage is that it can distribute video refresh
rate to each user. When the Unity3D engine operates at a
standard frame rate of 60Hz, the double-size stereo images
of two users should be rendered and output in each game
frame. Taking three users as an example, the rendering
sequence of n pairs of virtual cameras is as follows: In the
first frame, the stereo cameras of User 1 and User 2 conduct
rendering; in the second frame, the cameras of User 3 and
User 1 conduct rendering; and in the third frame, the cameras
of User 2 and User 3 conduct rendering, thus to form an
average and circular rendering sequence. If two projectors
are divided based on user number, resolution may be
distributed averagely only when user number is even, and for
3 or 5 users it cannot be distributed. Figure 7 shows virtual
camera rendering sequence and shutter open/close sequence
of each user.
As DPA adopts the mode of multi-user video projection in
order, and uses dedicated shutter to realize independent
watching, the user number can be extended to 6, but with an
image flicker caused by a 20 Hz refresh rate. The experiment
show that 3 or 4 is the optical number for this system.
Figure 7. Rendering sequence of virtual camera pairs and
corresponding open/close timing of shutter glasses.
APPLICATION EXAMPLES
This section introduces two application examples, with one
being a co-located multi-user virtual fireman simulation
training system employing DPA and the other a co-located
virtual tennis simulation system utilizing TPA.
Virtual Fireman Training System
A co-located virtual fireman training simulation system has
been developed (See Figure8 (a)). As three firemen typically
form a basic action group in real firefighting actions, three-
user mode is chosen for co-located collaborative fireman
training.
Here, DPA is selected to construct the system since 40Hz
refresh rate for each user can satisfy the requirement of
image quality for virtual roaming and extinguishing in the
fireman training system. It is constructed by two Benq
MS524 projectors connected by a NVIDIA Quadro P4000
graphics card. Obviously, the construction of DPA is simple
and has lower cost.
The Unity3D game engine is employed as the VR platform
to build the architectural environment, and particle effect is
used to simulate flame and smog to implement virtual fire
scene. The users watch the virtual fire environment from the
first perspective and three sets of 3D cameras are set in the
virtual scene, offering independent stereoscopic videos
rendering for three users and displaying in order. Three users
can watch their own video by the dedicated shutter glasses as
shown in Figure 8(a).
A virtual squirt gun has been designed as the interactive tool
for fireman users, which is easy for a user to control the move
path and the extinguishing action. Its outline is generated by
3D printing based on the actual structure. A rocker switch is
set on the virtual squirt gun to control the avatar’s movement
in a virtual scene, and a 9-axis inertial sensor is put inside the
CHI 2018 Paper CHI 2018, April 21–26, 2018, Montréal, QC, Canada
Paper 599 Page 6
squirt gun to gather the hand direction of the user, such that
the fireman user can identify the gun direction and set the
ejection button to eject virtual water. The virtual gun, by
means of Bluetooth, uploads the user’s movement control,
aiming direction, and ejection information to the server.
Based on the information, the server controls the movement
and fire control of three virtual avatars in a virtual
environment.
(a)
(b)
Figure 8. Application Examples: (a)Virtual Fireman Training
System, (b) Virtual Tennis Simulation System.
Virtual Tennis Simulation System
We also developed a virtual tennis simulation system with
TPA (See Figure 8(b)), since it maintains 60 Hz Monocular
refresh rate, which can be effective to display the high-speed
movement of the ball. Two of the users are players facing the
same screen watching the movements of their opponent’s
avatar and the virtual tennis ball from their own perspective.
The third one is an audience enjoying the game. These co-
located users can roam in a virtual environment dependently.
Such features are achieved with the help of the multi-user
display system.
Here, TPA is composed of three adjusted Benq MS524
projectors. It can achieve 360Hz refresh rate. We adjusted
two projectors by rotating their color wheels. The
modification is simple and easy. What is more, each adjusted
projector in the TPA can still be used as a color projector
alone after adjusting the setup of its output hue from the
projector setup menu or the graphics card menu. So the
incurred cost is very low too.
A Microsoft Kinect is adopted to capture the movements and
swing speeds of two players. According to these data, the
route and speed of the return ball is simulated and displayed
on the screen with TPA.
DISCUSSION
We further discuss and analyze the performance and
implementation of our systems in this section. Besides the
items listed in Table 1, we also discuss issues regarding
brightness decrease, frame rate decay, rainbow effect, and
potential crosstalk.
The number of projectors and users: Projection-based
multi-user stereoscopic display extends single viewpoint of
classical 3D projection technology to multiple viewpoints,
so that each user can watch an exclusive stereoscopic video
conforming to his or her position and view. The key issue
is to overcome the multi-way video overlapping projection.
When the user capacity of multi-user 3D display system
is n, 2n-way video is needed in the screen area. Although
optical polarization can only distinguish 2-way video,
active stereoscopic can upgrade user capacity through a
timesharing display, and double the number of users
combining with optical polarization. The optical polarization
is used in Riege [14], C1×6, TPA and DPA, but it plays
different roles. In Riege [14] and TPA (only needed when 2
TPA serve 6 users), it is used to distinguish two groups of
users, who perceive stereoscopic visions with active stereos.
In the C1×6 System and DPA, it is used to distinguish left
and right eyes, in which circumstance both left and right
images are projected to the screen simultaneously.
TPA can implement stereoscopic projection for three users
based on three projectors. Combined with the optical
polarization, it can also extend three-user arrays to six users,
resulting in more flexibility to users. On the other hand, DPA
is constructed based on two projectors without modification;
and with the help of dedicated shutter glasses we designed,
DPA can flexibly support two to six users.
Modification of hardware: As TPA only needs a simple
rotation to adjust the sequence of the color wheel of the
projector without any other modification, its construction is
much easier and simpler than the C1×6 System, resulting in
the lower cost of a TPA system. Additionally, since color
wheels in the C1×6 System are removed and a fixed color
filter is used in one of the primary colors, causing the inner
temperature out of limits and thus requiring extra cooling
modules; as a result, the modified projector cannot work on
its own. However, TPA only changes the color sequence of
the color wheels; thus it avoids the heating problem and each
adjusted projector can still be used as a color projector alone.
CHI 2018 Paper CHI 2018, April 21–26, 2018, Montréal, QC, Canada
Paper 599 Page 7
We design dedicated shutter glasses using the force-to-zero
driving strategy, which can reduce the switching time of the
LCD shutter, thus avoiding crosstalks at high refresh rates.
Since only two projectors and the dedicated shutter glasses
are needed, and no modification is necessary for the projector,
the realization of DPA is much easier and its cost is lower
than those of the C1×6 System and TPA. Therefore DPA can
provide more convenient environment construction for co-
located collaborative virtual environments.
Refresh rate: The refresh rate of the present popular
commercial 3D projector is 120Hz, showing 60 frame left-
eye image and 60 frame right-eye image per second. For
multiple users, the refresh rate per interval can be increased
by parallel projection of multiple projectors. A relatively
simple model is composed of two projectors for parallel
projection, combined with an optical polarization, to achieve
the 120Hz refresh rate for two users. When the number of
users is more than 2, the monocular refresh rate of each user
is scaled down; in this case, no physical or circuit
modification of the projector is necessary, but dedicated
shutter glasses are needed which is simple to implement, as
DPA does. Another model is to modify the projector
structure through the triple-projector parallel working to
increase the refresh rate to 360Hz and shorten the opening
time of the shutter glasses to 1/3, as done in TPA and the
C1×6 System. The refresh rate decay of DPA can lead to
image flicker to different extents: for 6 users at 20Hz the
flicker is noticeable; for 4 users at 30Hz the flicker is slight;
and for 3 users at 40Hz, the flicker is too subtle to perceive.
Thus serving 3 to 4 users with DPA will be a proper choice.
Crosstalk: In multi-user projection, the image contents of
two adjacent frames are not consistent, since they may be
perceived by different users from different perspectives.
Thus, crosstalk images are much more noticeable in multi-
user projection than those in a normal one. To avoid crosstalk,
high switch speed of LC shutter is required. Our force-to-
zero driving strategy of the LCD shutter reduces the
switching time effectively to lower the occurrence of
crosstalk. As we employ the force-to-zero driving strategy in
the design of the dedicated shutter glasses, the switching time
of the LCD shutter is decreased; as a result, DPA can avoid
crosstalk at a high refresh rate.
Perceived brightness: The perceived brightness of our
systems is evalued by having participants watching the same
virtual scene with a normal single view 3D projector and our
multi-user display systems. We adjust the image brightness
of our systems to make the participants obtain similar
brightness perception as the normal projection. For TPA, the
flow of light per interval is maintained with three projectors
working concurrently, while the perceived brightness is
decreased by roughly 10%, as the shutter glasses have a
lower duty cycle. For DPA, the duty cycle maintains, while
the lower refresh rate and the attached polarizer lead to
approximately 18% decrease in perceived brightness.
Rainbow effect: The rainbow effect is caused by the defect
of the single-chip DLP projector, which projects R-G-B bit
planes of the same image in sequence. As a result, the
audience who are extremely sensitive to the sense of color
may experience the asynchronization of colors. Statistics
show that the proportion of these people is quite small. In
TPA, the R-G-B bit planes are projected at the same time by
triple projectors to minimize the risk of the rainbow effect;
but there is still possibility of rainbow effect in DPA.
CONCLUSION
A multi-user stereoscopic projection display system extends
the single view to a multi-view display, which is an effective
display means to realize multi-user co-located collaboration.
Each user can watch the virtual scene through his/her single
correct perspective view based on his/her own position or
viewpoint while increasing their sense of reality experience
and immersion.
In this paper, two projection-based multi-user stereoscopic
display systems are introduced. One is called TPA, which is
based on a triple-projector array and can provide a 120Hz
active stereo for three or six users. The other is called DPA,
which is a dual-projector and easy-realized system providing
individual stereoscopic video stream for 2 to 6 users. Both
systems are flexible and easy to be implemented by
researchers or engineers. They can produce multi-view
systems supporting collocated collaboration such as the
multi-user virtual fireman training system and the virtual
tennis simulation system. The systems can also be used in
many other applications such as synchronously playing
PowerPoints with different languages and playing movies
with multi-language subtitles for multi-users.
Two problems remain to be settled to further improve the
multi-user stereoscopic projection display technology. One
is to enhance the refresh rate of the projector image. The
inner image refresh rate of certain types of projectors is more
than 120Hz. For example, the Panasonic PT-HZ900 type
LCD projector and the triple-chip DLP projector have a
refresh rate of 480Hz; and the double-speed DLP projector
with RGBRGB6 phase color wheel has a rate of 240Hz.
However, these projectors remain outputting with the refresh
rate of 120Hz. For instance, the Panasonic PT-HZ900
projects each image four times to perfect crosstalk and
residue shadow, while the 240Hz DLP projector projects
each image twice to improve the rainbow effect.
Theoretically, higher refresh rate of the above types of
projectors may be used to extend user capacity. The second
problem lies in the open/close switch rate of the shutter
glasses. If the number of users increases, the open/close
switch rate of the shutter should be improved in response,
and the experiment has revealed overdriving zero crossing
driving might be used for the open/close switch rate of LC
shutter to support 360Hz or higher. There is much room for
improvement regarding multi-user projection display.
CHI 2018 Paper CHI 2018, April 21–26, 2018, Montréal, QC, Canada
Paper 599 Page 8
ACKNOWLEDGMENTS
We would like to thank all reviewers for their valuable
comments. This work is supported by the China National
Key Research and Development Project (2016YFB1001403),
the National Natural Science Foundation of China under
Grant (61472225), the Shandong Provincial Science and
Technology Development Program (2016GGX106001), and
the China Postdoctoral Science Foundation (2017M620284).
REFERENCES
1. Maneesh Agrawala, Andrew C. Beers, Bernd Frohlich,
Pat Hanrahan, Ian McDowall, and Mark Bolas. 1997.
The two-user Responsive Workbench: support for
collaboration through individual views of a shared
space. In Proceedings of the 24th annual conference on
Computer graphics and interactive techniques, 327-
332. http://doi.org/10.1145/258734.258875
2. Barco. 2017. Virtual surgery table. Retrieved August
22, 2017 from http://www.barco.com/en
3. Stephan Beck, André Kunert, Alexander Kulik, and
Bernd Froehlich. 2013. Immersive group-to-group
telepresence. IEEE Transactions on Visualization and
Computer Graphics 19, 4: 616-625.
4. Wei Y. Chen, Nicolas Ladeveze, Céline Clavel, Daniel
Mestre, and Patrick Bourdot. 2015. User cohabitation
in multi-stereoscopic immersive virtual environment
for individual navigation tasks. In Virtual Reality (VR),
47-54. http://dx.doi.org/10.1109/VR.2015.7223323
5. Niranjan Damera-Venkata, and Nelson L. Chang.
2007. Realizing super-resolution with superimposed
projection. In Computer Vision and Pattern
Recognition, 1-8.
http://dx.doi.org/10.1109/CVPR.2007.383463
6. Sunkwang Hong, B. Berkeley, and Sang S. Kim. 2005.
Motion image enhancement of LCDs. In IEEE
International Conference on Image Processing, 11-17.
http://dx.doi.org/10.1109/ICIP.2005.1529980
7. Abhijit Karnik, Walterio Mayol-Cuevas, and Sriram
Subramanian . 2012. MUSTARD: a multi user see
through AR display. In Proceedings of the SIGCHI
Conference on Human Factors in Computing Systems
2541-2550.
http://doi.acm.org/10.1145/2207676.2208641
8. Alexander Kulik, André Kunert, Stephan Beck, Roman
Reichel, Roland Blach, Armin Zink, and Bernd
Froehlich. 2011. C1×6: a stereoscopic six-user display
for co-located collaboration in shared virtual
environments. ACM Transactions on Graphics 30, 6:
188.
9. Roman Lissermann, Jochen Huber, Martin Schmitz,
Jürgen Steimle, and Max Mühlhäuser. 2014. Permulin:
mixed-focus collaboration on multi-view tabletops. In
Proceedings of the 32nd annual ACM conference on
Human factors in computing systems 3191-3200.
http://dx.doi.org/10.1145/2556288.2557405
10. Jonathan Marbach. 2009. Image blending and view
clustering for multi-viewer immersive projection
environments. In Virtual Reality Conference 51-54.
http://dx.doi.org/10.1109/VR.2009.4810998
11. Wojciech Matusik, Clifton Forlines, and Hanspeter
Pfister. 2008. Multiview user interfaces with an
automultiscopic display. In Proceedings of the working
conference on Advanced visual interfaces 363-366.
http://doi.acm.org/10.1145/1385569.1385634
12. Brice Pollock, Melissa Burton, Jonathan W. Kelly,
Stephen Gilbert, and Eliot Winer. 2012. The right view
from the wrong location: Depth perception in
stereoscopic multi-user virtual environments. IEEE
Transactions on Visualization and Computer Graphics
18, 4: 581-588.
13. Tom Peterka, Robert L. Kooima, Javier I. Girado,
Jinghua Ge, Daniel J. Sandin, Andrew Johnson, Jason
Leigh, Jurgen Schulze, and Thomas A. DeFanti. 2007.
Dynallax: solid state dynamic parallax barrier
autostereoscopic VR display. In Virtual Reality
Conference 155-162.
http://dx.doi.org/10.1109/VR.2007.352476
14. Kai Riege, T. Holtkamper, G. Wesche, and B. Frohlich.
2006. The bent pick ray: An extended pointing
technique for multi-user interaction. In IEEE
Symposium on 3D User Interfaces 62-65.
15. Andreas Simon. 2005. First-person experience and
usability of co-located interaction in a projection-based
virtual environment. In Proceedings of the ACM
symposium on Virtual reality software and technology
23-30. http://doi.acm.org/10.1145/1101616.1101622
16. Jun Wang, Kyeongyuk Min, and Jongwha Chong.
2007. A hybrid image coding in overdriving for motion
blur reduction in LCD. In Entertainment Computing
263-270.
https://doi.org/10.1007/978-3-540-74873-1_32
17. Hong D. Yu, Hui Y. Li, Wei S. Sun, Wei Gai, Ting T.
Cui, Chu T. Wang, Dong D. Guan, Yi J. Yang, Cheng
L. Yang, and Wei Zeng. 2014. A two-view VR
shooting theater system. In Proceedings of the 13th
ACM SIGGRAPH International Conference on Virtual-
Reality Continuum and its Applications in Industry
223-226. https://doi.org/10.1145/2670473.2670497
CHI 2018 Paper CHI 2018, April 21–26, 2018, Montréal, QC, Canada
Paper 599 Page 9