Rendering Virtual Shadow Puppet TheatresCatherine Taylor and Ken Cameron, Department of Computer Science, University of Bath

cct43@bath.ac.uk, K.M.Cameron@bath.ac.uk

Motivation and Introduction

Shadow puppet theatres are a traditional tool in storytelling, particularly in countriessuch as China and Malaysia, where they have been named as pieces of ‘intangible her-itage’ by UNESCO [3]. However, the popularity of this art form has decreased signifi-cantly in recent years due to the complexity and skill required to produce a successfulperformance and an increased disinterest from children. In order to create a virtualshadow puppet theatre and help modernise this tradition, we have developed a novelmethod to render photo-realistic shadow puppets in real time. Perceptual testing of ouralgorithm has concluded that the images produced are perceived to be more likely to bereal than photographs of real shadow puppets.

Shadow Puppet Model

To determine an accurate model, describing the behaviour of a shadow puppet, wecarried out experimentation with a real shadow puppet theatre. Firstly, the relationshipbetween the area of the shadow and the distance from the light source was considered.

Figure: Effect of varying distance between light source and occluder on shadow size

The light source chosen was a point light and so the shadow produced is a perspectiveprojection of the puppet onto the screen. In addition, it was found that, due to globalillumination, as the distance between the puppet and screen increases, the shadow in-tensity decreases. To model this, we defined a distance function, α(D), where kd is aconstant and s the colour of the fully illuminated screen.

α(D) = minD(puppet, screen)

kd, s

(1)

In our experiments, an incandescent bulb was used as a light source, creating a shadowwith both inner and outer umbra and penumbra. This can be modelled by representingthe bulb as two light sources.

(a) Photograph of real shadow puppet (b) Shadow puppet model

Figure: Shadow sections: A - inner umbra, B - fully illuminated, C - outer umbra, D - inner penumbra, E -outer penumbra

The theatre screen is a matte surface and so the illumination, I , at each point on thescreen can be found using the Blinn-Phong lighting model [1], with ambient and diffusereflection components, where ki are constants, L is the light direction and N is thesurface normal.

Itotal = Ikcmax((N.L), 0)kb + kaI (2)

Method

Previous work on creating virtual shadow puppet theatres have been focused on control-ling the movement of the virtual puppet and have used texture mapping to model theappearance of the puppet or have created stylised shadows [3]. In our work, we focusedon rendering photo-realistic shadow puppets.

The real-time software was developed using the shadow puppet model and OpenGL.The virtual puppets were modelled by textured quads.

Figure: Examples of puppet textures

Two shadow maps were created by rendering the scene from the point of view of the innerand outer light source. In contrast to traditional shadow map methods, a perspectiveprojection was applied to the puppet to model the shadow projection. Fragments in theshadow map store both a depth term and an indicator to whether the point is shaded.

Figure: Finding the visibility function using a summed area table

There exists a bijection between the light incident on the point on the screen and theregion of intersection between the occluder plane, created from a shadow map, and lightfrustrum [2]. Thus, the shadow intensity at a point can be determined by integratingover the area of intersection. We approximate this integral by finding the ratio of shadedto total pixels in this region using summed area tables. Each entry in a SAT is equalto the accumulated sum of pixel values which fall above and to the left of the currentpoint.

Figure: Puppet texture and corresponding summed area table

The visibility information obtained from the summed area table, for each light source,is combined with the depth information, textured mapped screen texture and lightingmodel in a final fragment shader to produce the virtual puppet.

In our software, the puppet was animated by applying affine transformations to thevertices of the puppet quad.

Results

Figure: (a) photograph of a real shadow puppet, (b) ground truth image created using Monte-Carloray-tracing, (c) shadow puppet algorithm.

The real-time software can render still images in HD at a rate of 259.55fps and animationsat 51.87fps. 32 participants were shown pairs of images, consisting of photographs ofreal shadow puppets (R), computer generated ground-truth images (MC) and computergenerated images using our proposed algorithm (PA), and asked to select the imagesthey considered most likely to be real. A two tailed t-test, with 5% confidence interval,was used to analyse the significance of the observed mean number of PA images selected.

Image Pair Expected Mean Observed Mean Std dev t-value significant?PA + MC 1 0.813 0.738 -1.438 NPA + R 1 1.5 0.672 4.209 Y

The participants were unable to distinguish between the ground truth images and thoserendered using our algorithm. However, they believed that the images produced by ouralgorithm were more likely to be real than photographs of real shadow puppets.

Figure: Examples of virtual puppets rendered using our algorithm

The participants were also shown a pair (PA + R) of animations and asked to select theone which they believed was most likely to be real. The PA animation was selected 21%of the time, with t-value t = −3.293, suggesting the participants were able to tell thatthe rendered animation was computer generated.

Conclusion

The proposed shadow puppet rendering algorithm can be used to create realistic virtualshadow puppet theatres which can run in real-time. Future experimental studies shouldfocus on the material of the puppet with research carried out on the effect the colour,translucency and transparency of the puppet has on the shadow produced.

References

[1] James F Blinn, Models of light reflection for computer synthesized pictures, ACMSIGGRAPH Computer Graphics, volume 11, p192-198, ACM, 1997

[2] Elmar Eisemann and Xavier Décoret, Plausible image based soft shadows usingocclusion textures, SIBGRAPI’06, p155-162, IEEE, 2006

[3] Khor Kheng Kia and Yuen May Chan, A study on the visual styles of wayangkulit kelantan and its capturing methods, CGIV2009, p423-428, 2009