real-time non-photorealistic shadow rendering

Master of Science Thesis

Real-Time Non-Photorealistic

Shadow Rendering

by

Tamas Martinec

Supervisor: Jan Westerholm

Tampere, 2007

Abo Akademi University

Master’s programme in Software Engineering

Department of Information Technologies

Information Faculty

To Ildiko, Marcell and Tamas

Contents

Contents i

1 Non-Photorealistic Rendering 71.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71.2 Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

1.2.1 Two-dimensional techniques . . . . . . . . . . . . . . . . . . 81.2.2 2.5D NPR techniques . . . . . . . . . . . . . . . . . . . . . 101.2.3 Three-dimensional NPR techniques . . . . . . . . . . . . . . 10

1.3 Styles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111.3.1 Toon-shading, Cel-shading, Outlining, Stylized rendering . 121.3.2 Pen and Ink, engraving and line art . . . . . . . . . . . . . 121.3.3 Sketching and hatching . . . . . . . . . . . . . . . . . . . . 131.3.4 Painterly style . . . . . . . . . . . . . . . . . . . . . . . . . 131.3.5 Illustrations . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

2 Real-time Shadow Rendering 172.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172.2 Shadow concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182.3 Shadow maps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

2.3.1 Artifacts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192.3.2 Non-linear shadow maps . . . . . . . . . . . . . . . . . . . . 20

2.4 Shadow volumes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212.4.1 Silhouette edges . . . . . . . . . . . . . . . . . . . . . . . . 212.4.2 Inside-outside test . . . . . . . . . . . . . . . . . . . . . . . 222.4.3 Stencil shadow volumes . . . . . . . . . . . . . . . . . . . . 232.4.4 Performance . . . . . . . . . . . . . . . . . . . . . . . . . . 24

2.5 Hybrid algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . 252.6 Soft shadows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

2.6.1 Image based soft shadows . . . . . . . . . . . . . . . . . . . 252.6.2 Geometry based soft shadows . . . . . . . . . . . . . . . . . 28

i

ii CONTENTS

3 Hardware accelerated computer graphics 313.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313.2 Modern graphics pipelines . . . . . . . . . . . . . . . . . . . . . . . 31

3.2.1 Modeling transformations . . . . . . . . . . . . . . . . . . . 323.2.2 Illumination . . . . . . . . . . . . . . . . . . . . . . . . . . . 323.2.3 Viewing transformations . . . . . . . . . . . . . . . . . . . . 323.2.4 Clipping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323.2.5 Projection . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323.2.6 Rasterization . . . . . . . . . . . . . . . . . . . . . . . . . . 323.2.7 Display . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

3.3 Shader programs . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333.3.1 Vertex shaders . . . . . . . . . . . . . . . . . . . . . . . . . 333.3.2 Pixel shaders . . . . . . . . . . . . . . . . . . . . . . . . . . 34

3.4 Shader developing environments . . . . . . . . . . . . . . . . . . . 343.4.1 Rendermonkey . . . . . . . . . . . . . . . . . . . . . . . . . 353.4.2 FX Composer . . . . . . . . . . . . . . . . . . . . . . . . . . 36

4 A real-time NPR shadow rendering example 394.1 Concept . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 394.2 Developing vertex and pixel shaders . . . . . . . . . . . . . . . . . 40

4.2.1 A general frame application using shader programs . . . . . 404.2.2 Hatching . . . . . . . . . . . . . . . . . . . . . . . . . . . . 424.2.3 Shadow hatching . . . . . . . . . . . . . . . . . . . . . . . . 464.2.4 Distance based shading of a transparent shadow texture . . 49

5 Conclusion and future work 515.1 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 515.2 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

Bibliography 53

A NPR shadow pass vertex shader source code 59

B NPR shadow pass pixel shader source code 63

List of Symbols and Abbreviations 67

List of Figures 68

Abstract

Non-photorealistic rendering (NPR) is a branch of computer graphics where im-ages are purposefully created to be different than real images. Early NPR workmostly focuses on recreating artistic styles like pen and ink, watercolor or paint-ings, but NPR possibilities are a lot more diverse than just recreating humanartistic styles. By modifying or completely redesigning photorealistic graphicspipelines, completely new, unseen visual worlds can be created.

A relatively rarely seen area in NPR is NPR shadows. Since shadow renderingrequires substantial computing power, it is rarely in focus of NPR work. Withmore and more powerful computer hardware today it is possible to satisfy theresource needs of non-photorealistic renderings complete with shadows.

This thesis presents a real-time non-photorealistic shadow rendering solu-tion to emphasize the importance of shadows in NPR. Two Rendermonkey demoshaders are reused to combine a hatching shader with a shader that uses theshadow map algorithm to generate hard shadows. The shadowed area is repre-sented with a different hatching direction and the shadow is transparently blendedto the base hatching on the receiver surface.

1

Acknowledgements

I would like to thank Jan Westerholm for being my guide through the long journeyof writing this thesis. His remarks and advice were essential to complete mywork. I would also like to thank him for introducing me to the world of computergraphics and choosing me as course assistant as this opportunity provided meinsight in areas where I would have not gone by myself.

I would also like to thank Jukka Arvo for his remarks on the initial materialand for his help at choosing this interesting topic.

Furthermore I would like to thank Roy Mickos, Jarno Juntunen, Anne Aal-tonen and Seppo Kolari to support my studies besides working.

Finally I thank my family: Ildiko, Tamas, Marcell and my parents for theirsupport throughout the times of writing the thesis.

3

Introduction

Throughout human history there has been an urge to reproduce the images weperceive with our eyes. Starting from early cave paintings [Cav07] to recordingmotion pictures with high-end digital cameras we capture moments of the visualworld around us. The longing to recreate what we see made us drafting themlong before we could really create the images exactly. We made drawings thatreminded us of what we saw as well as we could. The results are usually far fromthe image that was formed on our retinas when we observed them. Sometimes itwould take an intelligent being to be able to recognize the original. The image isdifferent because either the tools we used are not capable of recreating the image,we do not remember well enough what we saw or our drawing technique is notperfect. It is also possible that we do not want to recreate the exact image. Wemight want to emphasize different aspects of the object of our drawing, makean abstraction of it, or depict features that are not visible otherwise but whichwe know are there because for example we examined it from several angles. Theresulting differences are visible throughout the history of art. Human art therefore– possibly except for the 1960s movement called photorealism – does not recreateexactly real world images. Human art is not photorealistic. [HF05]

Computer graphics is similar to human art in the sense that in the beginningit did not have the means for the exact recreation of the visual world around us.Early computers had low-resolution displays and scarce resources to achieve theseveral orders of magnitude higher resolution of the images drawn by photonsarriving at our eyes. Early computer images were therefore as far from photo-realism as prehistoric cave paintings. Nevertheless computer graphics had thetheoretical possibility that one day, if computers kept getting better at the samerate they were always doing – at an exponential rate – we will have the meansfor an exact recreation of the visual world around us. Today we are very close tothat goal. We now have computer generated images that can fool the untrainedeye to think that the image we see could be part of our reality. If computers geta magnitude faster still and if we solve the problem of content creation then anancient dream will become true for us: photorealistic virtual reality.

5

6 CONTENTS

A subtle but very important part of generating images by computers is therendering of shadows. Shadows give the viewer important guidance about theposition of objects. Calculating and displaying correct shadows in computergraphics images requires an amount of resources comparable to the image ren-dering itself. For this reason shadows were not displayed on rendered images fora long time. It also took substantial time to achieve hardware speeds where shad-ows could be calculated real-time. First hard shadows – with sharp transitionsbetween shadowed and not shadowed areas – then soft shadows – with smoothtransitions – become possible to be rendered fast enough.

Similarly to photorealistic rendering, non-photorealistic (NPR) rendering canbe enhanced substantially by adding shadows to the rendered images. Since re-sources just recently achieved sufficient speeds for photorealistic shadow render-ing, non-photorealistic shadows can be found quite rarely in applications today.As NPR itself, NPR shadow rendering is an area of research still in its infancy.

This thesis will sum up the basics of the research done in non-photorealisticrendering and shadow rendering until today and present a real-time non-photo-realistic shadow rendering example:

• Chapter 1 summarizes non-photorealistic image rendering processes: possi-ble software techniques – series of image generation or alteration algorithms– that can generate such images and styles that are assembled from thesetechniques and resemble some real artistic styles or are completely new.

• Chapter 2 describes shadow rendering approaches with a special focus onthe two major hard shadow algorithms: shadow mapping and shadow vol-umes. The chapter also introduces current soft shadow algorithms.

• Chapter 3 introduces the reader to the possibilities of hardware acceleratedcomputer graphics. The chapter includes a short summary of graphicspipeline components and description of the possibilities of pixel and vertexshader programs.

• Chapter 4 describes the development of a non-photorealistic shader pro-gram using two existing demonstration shaders. The shader programs willbe gradually built up from a skeleton program until the required effect isachieved.

• Chapter 5 draws conclusions about the work done and presents some di-rections for possible future work in this area.

Chapter 1

Non-Photorealistic Rendering

1.1 Introduction

On the road of getting from images depicted by character mosaics to picture-like, photorealistic computer generated images, the renditions were different fromrealistic images for a long time. Wire frame images, flat shaded three dimensionalrenditions, textured ray-traced images were all closer and closer to reality, but thecommon basic goal of the mainstream of computer graphics was to finally achievephotorealism. Apart from this general approach there were attempts to createimages that are not like photographs but for some reason deviate from them.The reason could be the recreation of artistic effects, special artistic styles, sometechnical reason like displaying several views of an object on one drawing, or toaccent some aspects of it. Those rendering techniques that purposefully differfrom photorealism are collectively called non-photorealistic rendering.

Non-photorealistic rendering is an area of computer graphics where imagesand animations are purposefully created to be different from real images.

The term rightfully received a lot of criticism for its awkwardness of defining asubject as not being something. Better sounding names like ”artistic rendering”and ”art-based rendering” have appeared but they either did not cover the wholespectrum of possibilities or did not get accepted throughout the field.

Non-photorealistic rendering creates images or animations that are not likereality. To achieve this several software techniques exist. These techniques arealgorithms or series of algorithms that generate images that are different fromreal images. These techniques or several of them applied after each other canbe used to achieve visual effects that are called styles because many of themresemble existing artistic styles like paintings and engravings. There are styles

7

8 CHAPTER 1. NON-PHOTOREALISTIC RENDERING

that use only one technique to achieve the desired effect, but many styles useseveral techniques applied after each other.

1.2 Techniques

According to the process of image creation, NPR images can be created usingeither two-dimensional or three-dimensional techniques. 2-D techniques usuallywork on photos or pre-rendered image pixels and apply a process that createsthe desired effect. Three-dimensional techniques use data that describe positionsand materials of objects in three-dimensional space and use processes that createan image of a view of the given objects from a certain location. Some stylesuse both two- and three-dimensional techniques as it is certainly possible to mixthem. For example a two-dimensional painterly style can be applied after athree-dimensional toon-shading phase. The same or close to the same effect canbe achieved with several techniques. More complex techniques give better ormore versatile results but they also require more resources. The possibilities arealmost endless. By changing the parameters of the processes a virtually infinitenumber of effects can be achieved and only a very small fragment of these havebeen discovered and used in human arts. Some of the yet unused effects that werediscovered during the creation of NPR effects can be seen in some experimentalanimations [WFH+97]. In the following subsections we will consider two andthree dimensional NPR techniques and evaluate their properties and merit.

1.2.1 Two-dimensional techniques

Two-dimensional techniques are performed on pixels of photos or pre-renderedimages. Such techniques are used for example to produce the painterly style orengravings. These techniques can be grouped according to the type of algorithmsthat are usually used to generate them. These groups will be described in thefollowings.

1.2.1.1 Halftoning, Dithering and Artistic Screening

In most of these techniques shades are created with methods used in printing.Printers create images with tiny dots of ink of different sizes that looks as contin-uous shades and shapes from a distance. The patterns used for these processescan be also reused to achieve different artistic or illustrational effects. Halftoninguses dot patterns to achieve continuous shading effects. Since printers use onlyblack, ink shades were created with arrangements of ink dots of different size sosmall that they could be only seen with a magnifying glass. Dithering was usedto display a wider color range with devices with a limited color production capa-bilities. Using special arrangements of pixels of different colors, the effects of new

1.2. TECHNIQUES 9

Figure 1.1: Artistic screening with non-linear screen element mapping [SS02]

colors or smoother shades can be achieved. These effects can be used for artisticor illustrational effects in NPR. In artistic screening meaningful motifs or pat-terns are used to generate the required shades and figures. Non-linear mappingof these motives can result in special warping effects of the perceived surface ofthe image (see Figure 1.1). Another technique belonging to this category is themethod of assembling pictures from images highly reduced in size used as pictureelements in a mosaic-like way.

1.2.1.2 Strokes

In the beginning the purpose of non-photorealistic rendering was to automati-cally recreate artistic effects on different images. Most of human art is created bypainting colors onto a two-dimensional medium with free hand strokes of a tool.These strokes are different from mathematically correct curves as the freehanddrawing capabilities of humans are limited by anatomy and technique. The dif-ferences can be summarized with the following features that can be implementedin algorithms that mimic the freehand style:

• Wiggliness: The curve randomly deviates from a mathematical smoothcurve.

• Length: The start and endpoint is different from the intended.

• Thickness and brightness: Random changes of width and intensity through-out the line.


• Line ends: The shape of line ends can be different than the rest of the line.(Small hooks or dots.)

Besides these main features several other attributes can be considered whendrawing artistic lines. There are a number of mathematical methods of describ-ing the lines as well. Between applying the traditional line drawing algorithmswith random deviations several times and strokes approximated by parametricpolynomial curves and projected onto three-dimensional bodies there are severalmethods that can be used depending on our purpose and the resources available.The line width and features of the line drawing tool also introduce additional pa-rameters. Using wavelet-theory it is possible to arbitrarily extend the resolutionof lines during magnification [FS94]. It also enables some new special effects inline drawing. Besides applying these artistic strokes in automatic image gener-ation these strokes directed by an artist in interactive painting applications canhelp the artistic process remarkably.

1.2.1.3 Simulating natural media and tools

The look of hand drawn and painted works of art can be best approximated bysimulating the interaction of natural materials and tools. To describe the inter-action of paper and pencil, canvas and brush, etc. there has been a number ofmodels developed by researchers that can be implemented with different level ofdetail. The simulation can be realized with cellular automata, modeling the be-havior of the ink solvent, fluid simulation algorithms, or by modeling the behaviorof small fragments of tool-material on the media [SS02].

1.2.2 2.5D NPR techniques

2.5 dimensional techniques use some extra information in addition to the two-dimensional picture pixels that will be modified. Such information can be aninfrared or x-ray image of the same subject or additional geometric informationstored in an image (also known as g-buffer), like pixel-viewpoint distance (z-buffer), object selection or shading enhancing graphical masks. The additionalinformation can be used to highlight areas of the image that are special forsome reason (for example local temperature deviations from the neighboring areasin a car engine can highlight problems). This technique also usable for edgehighlighting, shading, shadowing and algorithmic texturing among many others[SS02].

1.2.3 Three-dimensional NPR techniques

Three-dimensional NPR effects can be seen in video games and films most of-ten. These techniques render images of subjects based upon their geometrical

1.3. STYLES 11

and material information, mostly by modifying subprocesses of traditional three-dimensional rendering. Many NPR styles use one or several three-dimensionaltechniques to achieve their desired final look like toon-shading, illustrations ingeneral and the hatching style.

1.2.3.1 Edges

Compared to traditional rendering, non-photorealistic rendering introduces somenew problems in geometry processing. In art the artist depicts the subject withthe most characteristic strokes that are usually object outlines and intersections.When images are created from three dimensional models these prominent strokeswill be located at positions where triangles that form the models meet. Some ofthese meeting lines or edges form the outlines of objects. Calculating edges arenot needed in photorealistic rendering since every visible geometric informationis displayed. In NPR however sometimes only these prominent edges are drawnwhich require their calculation [SS02].

1.2.3.2 NPR lighting models

Photorealistic rendering uses the lighting model – a simplified way of calculatingthe brightness of pixels in the generated images – that approximates the physi-cal behavior of light in the best possible way considering the available resources.With non-photorealistic rendering we have the opportunity to use completelydifferent models to achieve the required similar or completely novel effect. Thepurpose of the lighting effect can be emphasizing, communication of more infor-mation, or to achieve artistic effects. One possibility for example is to compressand shift shading color to only a part of the shading range and use the unusedcolors for outlining or to describe other information. Instead of using shades thatare the result of illumination, other information can be used also, like one thatdescribes object temperature. Such lighting enables for example the analysis ofobject temperatures on a three dimensional model of the object from angles thatare not possible in reality [SS02].

1.3 Styles

A style in NPR graphics refers to the similarity to a real artistic style or artistictechnique of the image generated. Depending on what real artistic style the resultis similar to, NPR styles were grouped to sets named after real artistic styles.There is an unlimited number of non-photorealistic ways an image can be created.Most early NPR styles were efforts to recreate real artistic styles algorithmically.Some real styles were created with extremely tedious work that can be greatly


Figure 1.2: Objects shaded with real-time hatching effect. [PHWF01]

accelerated now with the help of computer automation. Let us examine the basicnon-photorealistic rendering style types.

1.3.1 Toon-shading, Cel-shading, Outlining, Stylized rendering

The earliest NPR styles were trying to achieve a cartoon-like style. This style isreferred to with many names such as toon-shading, cel-shading or stylized render-ing. The image is rendered with traditional three-dimensional rendering methodsbut during shading only a few discreet colors are used instead of the continuousshading of photorealistic rendering. Generally outlines of objects – just like incartoons – are emphasized at the same time for which several algorithms existwith different image quality and resource need. Although the result seems simplethe process requires more resources than traditional rendering since besides nor-mal rendering done with a limited number of colors, emphasizing edges requiresextra computational power [SS02].

1.3.2 Pen and Ink, engraving and line art

Images that are created by straight and curved lines placed against a usuallyplain background without gradations of shade or hue are called line art. Types ofline art include drawings made by pen and ink and also engravings. Engravingsare designs incised onto a hard, flat surface – traditionally a copper plate – bycutting grooves into it. The hardened steel tool that is used to cut the designsinto the surface is called a burin. The similarly named NPR styles reproduce theimage world of these artistic techniques. The intensity of lines is given by thetool used: it does not change or it can take only specific distinct values just astheir real counterparts. Basic line drawing algorithms can be used to recreateart with fixed width and color lines. More sophisticated line drawing algorithmsalso exist to better mimic lines drawn by these tools considering for example toolshape and tool angle with respect to the surface [Gas04].

1.3. STYLES 13

Figure 1.3: A frame from the Oscar winning film for best visual effects usingpainterly style: ”What Dreams May come” [REV07] (Copyright 1998 PolyGramFilmed Entertainment Distribution, Inc.)

1.3.3 Sketching and hatching

Sketching and hatching is considerably similar to the pen and ink style with thedistinction of the usage of continuous shades that are achieved by hatching meth-ods. The sketching style reproduces handmade pencil or pen sketches. Uncertain,broken, redrawn lines, uneven hatching and shading, simple motifs and drawingof only the main features characterize them. The direction and style of linesconveys the pictured material, tone and form. Hatching can be used to createshading with tools that otherwise would not support the creation of continuousshades like ink tools or engraving tools. Hatching in computer graphics can beimplemented with line drawing methods that require more resources but givebetter and more controllable results or with texturing that uses prepared imageswith a hatching pattern. (Figure 1.2)

1.3.4 Painterly style

The painterly style imitates the effect of painting on natural media – such aspaper, canvas, wood or glass – with a brush. The procedure usually takes strokespeed, pressure, type and angle of the brush and canvas properties into consid-eration. Some techniques model the interaction of paint particles and the canvassimulating the drying of the paint on the uneven surface of the canvas. Besidesgeneral painting effects, specific styles and techniques can be recreated to simu-


Figure 1.4: Technical illustration with ghosting view (Copyright 1994-2006 KevinHulsey Illustration)

late the style of a given painter like Monet, Van Gogh or John Mallord WilliamTurner. (Figure 1.3)

1.3.5 Illustrations

Illustrations are images that are created to decorate, describe or help to under-stand written text. Since the subject is more important than photorealism withthese images, computer generated pictures are quite frequently used as illustra-tions.

1.3.5.1 Scientific and medical

Scientific and medical illustrations were among the first types of illustrations.The methods used in medical illustrations are so commonly used nowadays thatsometimes we do not even notice them consciously, for example marking arterieswith red and veins with blue. NPR rendering techniques can accelerate the cre-ation of these extremely detailed images considerably. Besides different coloringschemes several shading methods, and projection types can be used, sometimeseven within the same image. Special projection types enable more views of thesame object from different angles on the same picture.

1.3.5.2 Technical illustrations

Technical illustrations include cutaway, ghosting (Figure 1.4), section and ex-ploded views of objects. Several computer aided design (CAD) tools support theautomatic creation of such technical illustrations. Cutaway views let us see theinner parts of object as if they were cut open. Ghosting images display outerlayers of objects transparent thus giving a better understanding of the insidecontents of objects. Exploded views display subjects taken apart, helping to un-

1.3. STYLES 15

derstand how the specific object is put together from smaller components. Sincethese illustration are rarely photorealistic their creation belong to NPR styles aswell. The features of these images are nowadays controlled by ISO standards.Some parts of these figures can be omitted, some can be emphasized for betterunderstanding or for highlighting features. The placement of symbols and labelscan also be automated with these tools.

Chapter 2

Real-time Shadow Rendering

2.1 Introduction

Shadows play an important role in our vision. The intensity and location of shad-owed areas help the viewer identify positions and spatial relations of objects inthree-dimensional space (see Figure 2.5). Calculating physically correct shadowswithin the time frame of change perceivable by a human eye is far out of the reachof current computer hardware. For applications of computer graphics where hu-man or other outside interaction changes the scene of the images rendered requirethat a complete image generation is executed within this time limit. Generallyif an algorithm has to finish within a pre-set time limit it is called a real-timealgorithm.

In computer graphics real-time rendering means that the rendering cycle hasto finish within the time limit of human perception of change – roughly 15ms.

Physically correct shadows require tracing light after its emission from thelight source, its reflection and absorption on all objects that the light falls on inthe scene until all of its energy is exhausted. Implementing such algorithms thatmodel this behavior results in hours of render times with currently used imageresolutions on available hardware. To be able to render images with shadowswithin the limits given in real-time computer graphics, our model has to belargely simplified.

In this chapter the two most common hard shadow rendering methods –shadow maps and shadow volumes – will be presented after describing basicshadow concepts. Also a method called hybrid algorithm is described that usefeatures of both basic algorithms. At the end of the chapter a short summary ofsoft shadow rendering algorithms will also be given.

17

18 CHAPTER 2. REAL-TIME SHADOW RENDERING

2.2 Shadow concepts

Shadowed areas are sections of space where objects partially or fully block thepath of light to that specific area. Point lights create hard shadows since thereare no such areas in lit surfaces where only a part of the light source is seen,therefore lit by only a section of the light source. If the light source is extended itcreates soft shadows where transitions from shadowed to lit areas are continuous.(Figure 2.6) The object blocking the path of light is called blocker or occluder,the object where shadowed and lit regions are created because of the occluder iscalled the receiver. The outline shape of an object with a featureless interior iscalled silhouette. Shadow shapes created on receivers are projected silhouettes ofblockers.

2.3 Shadow maps

One of the most widely used shadow algorithms is shadow maps, a concept in-troduced by Williams in 1978 [Wil78]. The shadow map is a two-pass algorithm.In the first pass depth values – the distance between the light source and oc-cluders – are rendered into a buffer called the shadow map of the scene from thepoint of view of the light source. In the second pass the image is rendered fromthe camera’s point of view using the shadow map to decide whether a pixel isin shadow or not. The decision can be made using the depth values. Williamstransformed the rendered pixel distance value into the light source coordinatesystem and compared the distance to the depth map value to decide if the pixelis in the shadow (later called backward shadow mapping).

Shadow maps are images that describe the distances of object pixels fromthe light source. By transforming these maps to camera space it can be decidedwhether rendered pixels are lit or are in shadow.

It is also possible to transform the shadow map into the cameras coordinatesystem and make the same comparison (forward shadow mapping [Zan98]). Bytransforming the shadow map to the camera coordinate space the depth valuesof the shadow map get comparable to the camera point of view depth map and acomparison gives as a result whether the pixel perceived by the observer is in theshadow or not. (Figure 2.1). The shadow map transformation can be executed us-ing hardware acceleration since most current hardware support projective texturemapping. Being the computationally cheapest real shadow rendering algorithmshadow maps are widely used in computer games and other real-time renderingengines.

2.3. SHADOW MAPS 19

Figure 2.1: Geometric configuration of the shadow map algorithm. Sample A isconsidered to be in shadow because the point on the receiving plane is fartherfrom the light source than the distance stored in the shadow map (which belongsto the blocker). In contrast, sample B is illuminated. [Cha04]

2.3.1 Artifacts

Since shadow mapping is created using a very limited model of real shadows ithas its artifacts (visible differences from real shadows). Two major problemsare present when using the shadow map algorithm: aliasing and self-shadowingartifacts.

2.3.1.1 Aliasing

The resolution of the shadow map introduces aliasing along shadow edges that canbe highly distracting when looked at closely (Figure 2.3). The shadow maps arebitmaps with usually smaller resolution than the screen, therefore when they areprojected their pixels may cover several screen pixels resulting in blocky shadowoutlines. To smooth these shadow edges several partial and full solutions havebeen published that require more or less resources [Bra03]. Rendering shadowmaps with higher resolution decreases aliasing but - since the shadow map isprojected to observer space - at certain screen configurations aliasing is quite


Figure 2.2: Bias related artifacts. (a.) Too little bias: incorrect self shadowingeffect - pixels falsely determined to be in shadow (b.) Too much bias applied:shadow boundary problem. (c.) Correct bias. [Cha04]

severe even with high resolution shadow maps.

2.3.1.2 Self-shadowing artifacts

The discreet nature of pixels also lead to the problem that the projected shadowmap pixels and camera viewpoint rendered pixels have different sizes. This causesincorrect self shadowing effects where pixels are falsely determined to be in theshadow. This is known as surface acne (Figure 2.2.). This artifact is also theresult of that the projected shadow map pixels and the calculated image pixelsdon’t have a one to one mapping: some shadowed pixels are calculated not to bein the shadow. To correct this a constant value called bias is usually added tothe shadow map value that shifts the shadow map so that the comparison willnot result in shadowed pixels when they are not there. Selecting the correct biasis not easy since too much bias can result in the removal of certain parts of theshadow [Cha04].

2.3.2 Non-linear shadow maps

The original shadow mapping algorithm covers a limited volume of the scenewhere correct shadows can be calculated since the shadow map is a rectangle.Therefore the procedure is not usable or is only usable in a limited way for pointlights. One solution to the problem is to use several shadows maps (6 to coverall directions from a point light source [Die01]) or to use non-linearly projectedfor example dual paraboloid shadow maps that can cover the whole sphere ofa point light with only two shadow maps [Bra03]. Non-linear shadow maps areonly presented in research papers at the moment, their usage is expected in thefuture as using them requires more resources than the use of linear shadow maps.

2.4. SHADOW VOLUMES 21

Figure 2.3: Shadow map aliasing: (a.) 256x256, (b.) 512x512, (c.) 1024x1024shadow maps. [Cha04]

2.4 Shadow volumes

A more accurate shadow rendering method is the shadow volume algorithm.Shadow volumes enable precise shadow calculations but they also require a lotmore computational power than shadow maps. Shadow volumes describe shad-owed volumes of the scene using calculated geometry, instead of pixels.

Shadow volumes are geometrically determined sections of the scene space thatare used to determine whether objects and thus pixels of the rendered images liein shadow.

After creating shadow volumes it is possible to exactly decide about each pixelof the rendered image whether the surface it depicts is in the shadow or not. Theoriginal shadow volume algorithm [Cro77] had numerous limitations that mostlyhad been resolved since then [Bra03]. Also several suggestions have been made toaccelerate shadow volume calculations with current hardware features [Bra03].

2.4.1 Silhouette edges

Creating the geometry of shadow volumes is done by identifying silhouette edgesfrom the point of view of the light source and extending the silhouette to infinityaway from the light. The result of the operation is a semi-infinite volume thatdescribes the section of space that the given object creates a shadow in. There areseveral published methods of deciding whether an edge in an object is a silhouetteedge or not. The most simple way is to check if the silhouette edges are connectingfront and back facing triangles with respect to the light source. The orientationof the triangles can be decided by calculating the dot product of the face normaland a vector to the light source. A positive product indicates front facing, anda negative product indicates back facing triangles. By repeating this calculationfor all connecting triangles a set of silhouette edges can be obtained that forma closed loop. These calculations require that the objects are such polygonalmodels that are oriented (the face normals point away from the object), and


Figure 2.4: Inside-outside test: Counting intersections with shadow volumes (in-crementing at front facing polygons and decrementing at back facing polygons)give zero for non-shadowed pixels and non-zero for shadowed pixels.

watertight implying that any point in space is unambiguously inside or outsidethe object.

By extending the silhouettes far away to infinity from the light source semi-infinite volumes can be formed that describe regions in space which are in theshadow with respect to the given light source.

2.4.2 Inside-outside test

After shadow geometry has been calculated we have to decide about each renderedpixel if it lies within a shadow volume or not. The simplest algorithm for this isto follow a ray from the camera to each rendered pixel and count the crossings ofshadow volumes [Cha04]. If a front facing shadow polygon is crossed a counter isincremented and if a back facing shadow polygon is crossed it is decremented. Ifthe ray intersects an object the counter is tested for the value zero which impliesthat it lies outside of all shadow volumes and therefore is not in shadow. If thecounter is non-zero then the pixel is part of a shadowed area of an object. (Figure2.4.)

This method works in most scene configurations but there are cases when the

2.4. SHADOW VOLUMES 23

Figure 2.5: Shadow volumes: Final image (left), Silhouette edges (middle),Shadow volumes (right) [Bra03]

result is incorrect. If the camera is within a shadow volume or is clipped againstthe near plane of a shadow volume some regions will be calculated with invertedshadows.

2.4.3 Stencil shadow volumes

The inside-outside test for shadow volumes can be accelerated using hardwarestencil buffers [Hei91] enabling the method to be used in real-time applications.There are several approaches of the implementation: depth pass, depth fail, splitplane and exclusive-or.

2.4.3.1 Depth pass or z-pass

In the depth pass method only shadow volume polygons between the camera andobject are counted. Depth pass means that for these polygons the depth testwill pass and they will be used. In modern graphics hardware a bitmap calledthe stencil – generally used for stencil-like effects – can be used for operations inthe process of calculating the final images. In the depth pass method the stencilimage is calculated to describe shadowed and non-shadowed areas of the image.The stencil is calculated in two passes. First the front faces of the shadow volumesare rendered, each incrementing the stencil buffer values, then in the second passback faces are rendered decrementing the buffer values. After the operations thestencil buffer will have zeroes at non-shadowed pixels. This method fails in thesame situations as the inside-outside test, resulting in incorrect stencil valueswhen the camera is in shadow or if a shadow volume is clipped against the nearplane.


2.4.3.2 Depth fail or z-fail

The depth fail method solves the problems associated with the depth pass methodby inverting the stencil count scheme so that volumes behind the object belongingto the given pixel are counted, (for these polygons the depth test will fail duringrendering). To accomplish this successfully the rear end of shadow volumes haveto be capped. This method is also known as Carmack’s reverse [Bra03].

2.4.3.3 Split plane

The split plane algorithm [Lai05] uses either the depth pass or the depth failmethod for each tile of the rendered image based on comparing the contents ofa low-resolution depth buffer and an automatically constructed plane called thesplit plane. The location of a split plane is calculated from the scene configura-tion. It was shown that this method results in considerable fillrate savings on asoftware rasterizer, however, its hardware acceleration would require new hard-ware features namely being able per image tile to test against the selected splitplane.

2.4.3.4 Exclusive-Or

Instead of having the stencil buffer value as a counter it is possible to describe theinside-outside counter ray with a one bit value as being in or outside a shadowvolume. Using exclusive-or on a stencil buffer bit when crossing a shadow volumepolygon instead of addition and subtraction results in this bit describing whetherhe pixel is in the shadow or not. This method gives incorrect results when thereare intersecting shadow volumes but saves fillrate in case of a scene withoutintersecting shadows.

2.4.3.5 Stencil value saturation

For all stencil methods it is possible that the stencil value saturates if there are notenough bits to describe it, resulting in incorrect stencil values. The value whenstencil value saturation occurs depend on the number of bits used to describethe value. With the 8-bit stencil buffers used with current graphics hardware theproblem is unlikely.

2.4.4 Performance

With increasing geometric complexity shadow volume calculations require moreand more computational resources since shadow volumes are usually large, over-lapping polygons and each of them requires stencil updates. Several papers havebeen published that try to decrease the number of shadow volumes processedbased on for example discarding volumes outside a rectangle or a given depth

2.5. HYBRID ALGORITHMS 25

range. The most successful fillrate saving techniques – described in the followingsection – are hybrid algorithms that use shadow maps to determine shadow edgepixels for shadow volume calculations [Cha04].

2.5 Hybrid algorithms

Hybrid shadow algorithms use the advantages of both shadow maps and shadowvolumes. During the execution of these methods shadow maps are used to de-termine what areas in the picture contain shadow edges and shadow volumecalculations are executed only for those pixels of the image. With these methodshighly accurate shadows can be calculated considerably faster than using onlyshadow volumes.

The first hybrid algorithm [McC00] used the shadow map to identify occludersilhouette edges with an edge detection algorithm. Edge detection algorithmsprocess bitmaps and mark points in a resulting image where the intensity ofthe image changes sharply. Based on the edges detected shadow volumes arereconstructed and used to render precise shadow edges.

Govindaraju et al. [GLY+03] first used special techniques to decrease thenumber of potential occluders and receivers and then rendered shadow edgesusing both shadow maps and object-space clipping.

Chan [Cha04] used edge detection to identify shadow edge pixels from a com-putational mask – a bitmap with information about the location of edge pixelsin the image – to calculate only shadow edge pixels on the stencil buffer.

2.6 Soft shadows

The previous methods are capable of calculating only shadows of points lights,or hard shadows. If light is emitted from a source with nonzero area, shadowsilhouettes become blurry and also significantly harder to render since betweenthe lit and the shadowed areas (umbra) there is an area that is lit by only a part ofthe light source (penumbra), see Figure 2.6. Most models for generating these softshadows have been so complex that including them in a graphics pipeline wouldcertainly make them so slow that real-time applications would be impossible.However with the ever continuing acceleration of graphics hardware some modelshave become feasible. It is possible to extend both shadow maps and shadowvolumes to generate soft shadows, each method inheriting the advantages anddisadvantages of the original.

2.6.1 Image based soft shadows

Image based methods are soft shadow generation procedures that are based on theshadow map algorithm. All of these methods create a shadow map that describe


Figure 2.6: Soft shadows [HLHS03]

shadowed areas and are used to calculate the color of the shadowed surfaces.Shadow maps for soft shadows contain several values per shadow map pixel to beable to represent continuous transitions from shadowed to non-shadowed areas.The following sections describe the main image base methods.

2.6.1.1 Combination of several point-based shadow maps

If hard shadows are calculated using sample points of the extended light sourceand the resulting shadows combined to a final image an approximation can beachieved that models real soft shadows accurately if the number of samples issufficiently high. This method requires that the shadow algorithm is executedas many times as samples are required resulting in very high rendering times,[HLHS03] (Figure 2.7).

2.6.1.2 Layered Shadow Maps

The layered method is an extension of the previous method. Instead of calculatinga shadow map for each object, one layered attenuation map is calculated for thewhole image that contains all objects that are visible from at least one lightsample point and describes the distance to the light source and the percentageof sample points seeing this object. The shadow map in this method containsas many layers of information as many samples are taken from the light source.The layers describe blocking entities between the light source and the receivers.During rendering the rendered pixel is checked if it is in the layered shadow map,

2.6. SOFT SHADOWS 27

Figure 2.7: Combining several hard shadows. Left: 4 samples. Right: 1024samples [HLHS03]

and if it is there its color is modulated according to the visibility percentagevalue in the map. Since the algorithm has to run as many times as the number ofsamples with this method also, rendering times are not acceptable for real-timeapplications on current hardware. [KM99]

2.6.1.3 Quantitative Information in the Shadow Map

Heidrich [HBS00] extended the shadow map method by calculating the percentagevalue of the light source visible for each shadow map pixel. Compared to thelayered shadow map method, there is only one shadow map in this approach,and this method does not involve the depth information in the shadow mapcalculation. Therefore this method calculates less accurate but faster computableshadows. The original algorithm worked only for linear light sources but later itwas extended by Ying [YTD02] to be usable for polygonal light sources.

2.6.1.4 Single Sample Soft Shadows

With the single sample soft shadow method a standard shadow map is calculatedusing a point light source and during rendering the depth map is analyzed toidentify occluder and receiver distances. These distances are used then to cal-culate a light attenuation coefficient that help determine the level of shadowingat a given location thus the size of inner and outer penumbra (see Figure 2.8).This way effects of area light sources are approximated using a point light. Themethod approximates soft shadow effects nicely considering the required amountof resources.


Figure 2.8: Inner and outer penumbra calculated from point light. [HLHS03]

2.6.1.5 Convolution technique

Soler et al. [SS98] observed that soft shadow calculations can be approximatedwith a convolution operation on the image of the light source and the image of theoccluder. The technique use the fast Fourier transform to efficiently convolute animage of the light source with an image of the blocker, and the resulting blurredblocker image is projected onto receiver objects. This method works well witharbitrary shaped light sources and produces high quality soft shadows at a limitedoperation complexity.

2.6.2 Geometry based soft shadows

Geometry based soft shadow methods use modifications of the shadow volumealgorithm to calculate soft shadows. Similarly to the shadow map based soft shad-ows it is possible to combine several shadow volumes taken from point samplesof the light source. Other approaches extend the shadow volume using a specificheuristic: Plateaus or Smoothies. It is also possible to compute a penumbra vol-ume for each edge of the shadow silhouette. In the following subsections thesemethods will be described in more detail.

2.6.2.1 Combining several hard shadows

Similarly to the combination of images calculated with the shadow map algorithmit is possible to average images calculated with shadow volumes. The number ofsamples required for good quality images are quite high requiring multiple timesthe resources of the original shadow volume algorithm.

2.6.2.2 Soft Planar Shadows Using Plateaus

With the plateaus method an attenuation map is generated using shadow vol-umes. The silhouette vertices of the objects are transformed into cones and

2.6. SOFT SHADOWS 29

surfaces and the resulting volumes are projected onto the receiver and penumbravolumes are colored using Gouraud shading. The method is limited to planarreceivers, it assumes a spherical light source and computes outer penumbra (seeFigure 2.8) only resulting in limited accuracy.

2.6.2.3 Smoothies

Smoothies are planar surfaces calculated from object silhouettes that describesoft shadow edges. The method can be fully accelerated with graphics hardware.Similarly to the plateaus method smoothies compute only the outer penumbra.Occluders therefore always project an umbra, even with large light sources withrespect to the occluders [Cha04].

2.6.2.4 Penumbra wedges

Another soft shadow method that is designed to be computed with the use ofshader programs is penumbra wedges. For each silhouette edge a silhouette wedgeis calculated. In a first pass normal shadow volumes are used to render a hardshadow. Then the wedges are used to calculate the percentage of the light sourcevisible for soft shadow pixels. The resulting visibility buffer is used when ren-dering the scene with standard methods. Penumbra wedges result in physicallyexact shadow in most cases[AAM03]. Compared to smoothies the difference isin the algorithm of calculating the soft shadow (or transparency) values of thepenumbra: smoothies calculate these values by extending hard shadows with pla-nar surfaces while penumbra wedges calculate these values by projecting the lightsource shape to the shadowed surface. Since the penumbra shape is more exactwith penumbra wedges this method provides even more exact results.

Chapter 3

Hardware accelerated

computer graphics

3.1 Introduction

The computational power of computer hardware has been accelerating exponen-tially since the invention of the integrated chip. In order to achieve faster andbetter looking three-dimensional graphics applications graphics cards were ex-tended in the middle of the 1990s by a dedicated processing unit: the GraphicalProcessing Unit (GPU). Before that graphics applications were handled entirelyin the main processor unit. Graphical Processing Units and their programminginterfaces have been continuously evolving. Three-dimensional image generationis divided into separate stages generally described by Foley et al. [FvDFH96]which is usually referred to as the graphics pipeline. Applications using thegraphics hardware usually access its features via a standardized application pro-gramming interfaces (API). The APIs provide uniform methods to access variousgraphics hardware thus supporting fast application development. Current widelyaccepted APIs for graphics hardware are DirectX and OpenGL. Each API hasits own implementation of the graphics pipeline.

3.2 Modern graphics pipelines

Data structures that describe three-dimensional scenes are fed to the graphicspipeline as its input and a two-dimensional view of the described scene is pre-sented as output. The stages of data processing can be summarized as follows:

31

32 CHAPTER 3. HARDWARE ACCELERATED COMPUTER GRAPHICS

3.2.1 Modeling transformations

The modeling transformations put objects to their designated place in the scenewith the correct scaling and rotations. Objects generally are described by trian-gles with the coordinates of their vertices. The output of the modeling transfor-mation are the same objects but with their vertex coordinates (spatial positionsof the points that define the triangles) changed according to the required trans-formations.

3.2.2 Illumination

The illumination phase calculates vertex colors according to the given lightingmodel. Current hardware graphics pipelines calculate lighting for vertices only,per pixel lighting values can be calculated with approximations by pixel shaderprograms in the rasterization phase.

3.2.3 Viewing transformations

The viewing transformations changes the position, rotation and scale of the ob-jects as they would be perceived from the camera position. This also includesperspective or orthographic projection according to the given settings.

3.2.4 Clipping

The clipping phase removes objects hidden from the camera from the process inorder to accelerate the remaining stages. This stage may include view frustrumculling, face culling or hidden object removal from the view frustrum.

3.2.5 Projection

Projection transforms all geometry to their 2D position on the camera screen.

3.2.6 Rasterization

The rasterization stage calculates the color of every pixel in the image based onthe previously calculated geometry and lighting information.

3.2.7 Display

The display phase deals with putting the rendered image to its final place (coor-dinates) on the display device - for example a window - and also handling possiblelow level issues of displaying the image like handling the set-up and special re-quirements of the display device.

3.3. SHADER PROGRAMS 33

Figure 3.1: A possible hardware rendering architecture including vertex and pixelshaders. [SL04]

3.3 Shader programs

Early 3D accelerator hardware functionality was fixed, using a simple subsetof the graphics hardware available today. The possibilities ended with settingparameters of the predefined graphics pipeline and specifying what textures touse on the geometry. Later as graphics cards became more and more sophisticatedarbitrary (though limited length) programmability became possible for some ofthe pipeline stages. Several different languages were proposed for programmingthese aspects of the graphics pipeline. Currently programmers can choose fromHLSL (High Level Shader Language), Cg (C for Graphics), or GLSL (OpenGLShading Language) as higher level languages or directly use the assembly-likelow-level instruction sets of the pipeline APIs.

3.3.1 Vertex shaders

Current parts of the graphics pipeline that deal with geometry modificationsare programmable with vertex shader programs. Modeling, viewing and pro-jection transformations along with any special geometry modification are freelyprogrammable today.


Vertex shader programs specify how graphics cards process geometry data.

Having free access to the geometry processing of the pipeline enables the pro-grammer to achieve visual effects that were not possible before. These effects in-clude procedural geometry, animation key-frame interpolation, particle systems,real-time modification of the perspective view and special lighting models.

3.3.2 Pixel shaders

Vertex shader functionality was in a limited way possible with fixed pipelines.Pixel shaders however enable possibilities that were previously unimaginable inreal-time applications. The word shader originates from the non-real time com-puter graphics industry. In ray-tracing applications shaders are separate softwaremodules that are passed all available geometrical and environmental informationabout a pixel and they return the color of that pixel. In order to achieve a similarlevel of realism as ray tracing graphics hardware adopted the same functionalitywith the name of pixel shaders.

Pixel shaders are programs that calculate the color for each displayed pixel.

Pixel shaders enable per-pixel non-standardized lighting effects that bring alevel of realism to real-time graphics that was not possible before. With pixelshaders non-photorealistic rendering also entered the real-time rendering world.Effects like procedural textures, volumetric effects, anisotropic lighting, amongothers, became possible.

3.4 Shader developing environments

Early shader development required extreme low-level understanding of the graph-ics pipeline APIs. Shader programs were written in an assembly like shader lan-guage that usually were compiled using simple tools provided with the APIs.Later higher level, C-like programming languages appeared and also the first In-tegrated Development Environments were developed by major hardware vendors.Also more visual tools appeared on the marked enabling shader development andexperimentation by connecting shader program elements visually as boxes ona graphical interface. Both of the major vendors ATI and NVIDIA has its ownshader developer IDE, that works best with their own products. With these toolsthe user can focus on shader development only, the rest of the pipeline API setup is done by the IDE. It also accelerates work with shaders as the componentsof shaders are visually presented and can be changed easily. In the followingsubchapters the IDEs of the market leaders in the graphics card industry will bedescribed.

3.4. SHADER DEVELOPING ENVIRONMENTS 35

Figure 3.2: Screenshot of ATI’s Rendermonkey shader development environment.

3.4.1 Rendermonkey

The shader developer IDE of ATI is Rendermonkey [RMD07]. Rendermonkeyversion 1.6 was used for the example shader development in this thesis. It sup-ports all shader models provided with DirectX 9.0c (including HLSL) and forshading using OpenGL GLSL shading language.

The motivation for developing RenderMonkey is given by ATI as follows[RMD07]:

• A powerful programmer’s development environment for creating shaders.

• A standard delivery mechanism to facilitate the sharing of shaders amongstdevelopers.

• A flexible, extensible framework that supports the integration of customcomponents and provides the basis for future tools development.

• An environment where not just programmers but artists and game designerscan work to create ”mind-blowing” special effects.


• A tool that can easily be customized and integrated into a developer’sregular workflow.

3.4.2 FX Composer

FX Composer is the shader developer IDE developed by NVidia [FXC07]. FXComposer empowers developers to create high performance shaders in an in-tegrated development environment with a real-time preview and optimizationfeatures. FX Composer was designed with the goal of making shader develop-ment and optimization easier for programmers while providing an intuitive GUIfor artists customizing shaders for a particular scene.

FX Composer supports all the standard features one would expect in anintegrated development environment for high performance shaders [FXC07]:

1. Creating shaders in a high powered development environment:

• Sophisticated text editing with intellisense (auto-complete) and syntaxhighlighting

• Work directly with HLSL .FX files, creating multiple techniques andpasses

• Use of the .FX files created with FX Composer directly in applications

• Convenient, artist-friendly graphical editing of shader properties

• Supports Microsoft DirectX standard HLSL semantics and annota-tions

• Support for all DirectX 9.0 shader profiles

• Developing shaders on models with lighting and animation

2. Debugging shaders with visual shader debugging features:

• Interactive compiler helps finding and fixing problems

• Visible preview of source and intermediate textures targets

• Interactive jump-to-error feature helping to locate problems quickly

3. Tuning shader performance with advanced analysis and optimization:

• Enables performance tuning workflow for vertex and pixel shaders

• Simulates performance for the entire family of GeForce FX GPUs

• Capture of pre-calculated functions to texture look-up table

• Provides empirical performance metrics such as GPU cycle count, reg-ister usage, utilization rating, and FPS.

• Optimization hints notifying of performance bottlenecks

3.4. SHADER DEVELOPING ENVIRONMENTS 37

Both Rendermonkey and FX Composer provide a completely set up 3D enginefor displaying shaders so developers can focus on shader development withoutworking through the tedious graphics pipeline setup and the quite steep learningcurve of fully programming the pipeline APIs.

Chapter 4

A real-time NPR shadow

rendering example

In this chapter the development of a non-photorealistic shadow rendering shaderwill be presented. The set of possible NPR techniques is vast, compared to photorealistic rendering - the extremely various visual world as we see it.

The basic details and possibilities of 3D rendering and shader programmingwill not be described in this thesis. Instead, these can be located for example inthe book Shaders for Game Programmers and Artists by Sebastien St-Laurent[SL04].

Using shader programs running in graphics hardware for our example effect,render times well within human perception can be achieved with current desktopcomputers. Therefore our example will be a real-time rendered NPR shadoweffect.

4.1 Concept

The NPR effect chosen is a hatching shader with the extension of presenting theshadows in the scene with a different hatching pattern than the object hatchingjust like an artist drawing the scene would present shadows with another directionof hatching lines. The geometry the effect will be presented on is a trefoil knotobject on a disc. The scene will be lit by a point light from above therefore thetrefoil knot will cast its shadow on the disc.

To implement the effect the following three rendering passes will be required:

• Object rendering pass: an NPR hatching pass

39

40 CHAPTER 4. A REAL-TIME NPR SHADOW RENDERING EXAMPLE

Figure 4.1: Screenshots of the reused Rendermonkey example shadersNPR/Hatch (left) and Shadows/Shadowmapping (right).

• Depth map rendering pass: rendering the shadow map from the point ofview of the light source.

• Shadowed area rendering pass: using the projected shadow map to decideif a pixel is in shadow. Unshadowed areas will be rendered with normalhatching and shadowed areas with normal hatching combined with a dif-ferent angle hatching pattern.

The first two passes can be reused from Rendermonkey example shaders calledNPR/Hatch and Shadows/Shadowmapping. The rest of the chapter will focuson the development of the third pass shaders.

4.2 Developing vertex and pixel shaders

4.2.1 A general frame application using shader programs

When creating a new pass in Rendermonkey four items are created automaticallyin the workspace tree:

• Model

• Vertex shader

• Pixel shader

• Stream mapping

The model can be any geometry in .3ds, .nms, .obj, or .x data formats [SL04].Let us choose the disc.3ds object from the Rendermonkey example objects asthe platform where the shadow of other objects will fall on. The file containsgeometry information about the disc: location of its points in space described by

4.2. DEVELOPING VERTEX AND PIXEL SHADERS 41

their spatial coordinates (x,y,z), and what points are connected by the surfacesthat define the object.

4.2.1.1 Skeleton vertex shader

A skeleton vertex shader is created with the new pass:

float4x4 matViewProjection ;

struct VS INPUT

{f loat4 Pos i t i on : POSITION0 ;

} ;

struct VS OUTPUT

{f loat4 Pos i t i on : POSITION0 ;

} ;

VS OUTPUT vs main ( VS INPUT Input )

{VS OUTPUT Output ;

Output . Pos i t i on = mul( matViewProjection , Input . Pos i t i on ) ;

return ( Output ) ;

}

The skeleton vertex shader consists of the declaration of the input and outputdata structures and the main function that applies the projection matrix to theinput geometry.

4.2.1.2 Skeleton pixel shader

A skeleton pixel shader is also created:

f loat4 ps main ( ) : COLOR0

{return ( f loat4 ( 1 . 0 f , 0 . 0 f , 0 . 0 f , 1 . 0 f ) ) ;

}

The pixel shader returns constant red color for all pixels that the shader isrun on. (The float values in the returned data structures stand for the red, green,blue and alpha values in the range of 0 to 1.)

Using a trefoil knot object the assembled 3 pass shader will result the imagepresented in figure 4.2. The disc object is intersecting the trefoil knot object thatcan be fixed in the vertex shader by offsetting the geometry with constants. Thevalues of the required constants can be easily determined since RenderMonkeyenables the user to dynamically adjust parameters during preview rendering. Twovariables are added and the value can be determined by adjusting them while theshaders are running:

f loat posShi f tY ;

f loat ob j e c tS c a l e ;


Figure 4.2: Disc object intersecting the trefoil knot object (left), fixed disc scaleand position (right).

// s c a l i n g model

In . Pos . xyz = ob j e c tS c a l e ∗ In . Pos . xyz ; // 1.6 i s f i n e

// s h i f t i n g p la t form model under the o b j e c t

Input . Pos i t i on . y −= posShi f tY ; //40 i s f i n e

4.2.1.3 Stream mapping

Stream mapping defines in what form geometry information is passed to thevertex shaders. The data is passed in data structures. It is possible to select inRendermonkey what information to pass to which hardware registers and how thedata is represented. The default stream mapping contains position informationonly, passed in a data type that holds 3 float values (FLOAT3).

4.2.2 Hatching

To add a hatching effect to the disc the same code can be added as for the hatchedobject.

4.2.2.1 Vertex shader

In the vertex shader, based on a diffuse lighting values, weights are calculatedfor each of the hatching textures. There are 6 hatching textures with differentshades. The calculated weights are passed to the pixel shader. To be able tocalculate the diffuse lighting a normal vector of the geometry is required as aninput parameter of the vertex shader. Also to be able to address the hatchingtexture pixels in the pixel shader, a texture input parameter is also added to theshaders and the unmodified value is passed to the pixel shader.

f loat posShi f tY ;

f loat ob j e c tS c a l e ;

f loat time 0 X : register ( c11 ) ;


f loat4 l i gh tPos : register ( c4 ) ;

f loat4 l i g h tD i r ;

float4x4 matViewProjection ;

float4x4 pro j mat r ix : register ( c6 ) ;

struct VS INPUT

{f loat4 Pos : POSITION0 ;

f loat3 Normal : NORMAL0;

f loat2 TexCoord : TEXCOORD0;

} ;

struct VS OUTPUT


f loat3 l i ghtVec : : TEXCOORD0; // to be used l a t e r

f loat4 shadowCrd : : TEXCOORD1; // to be used l a t e r


f loat3 HatchWeights0 : TEXCOORD3;


} ;

VS OUTPUT vs main ( VS INPUT In )

{VS OUTPUT Out ;

// Fl ip , s c a l e and t r a n s l a t e our model to s u i t our scene

// In r ea l app l i c a t i on s , t h i s work shou ld normal ly be

// done at load time , a l t e r n a t i v e l y i f animation i s des i red ,

// be a l t e r e d by a world matrix .

In . Pos . xyz = ob j e c tS c a l e ∗ In . Pos . xyz ; //was 1.6

In . Pos . y −= posShi f tY ; //was 40

// Input . Normal = Input . Normal . xyz ;

Out . Pos = mul( matViewProjection , In . Pos ) ;

Out . TexCoord = In . TexCoord ;

// Animate the l i g h t po s i t i on .

// Comment out t h i s code to use a s t a t i c l i g h t .

// In r ea l a p p l i c a t i o n s t h i s work i s b e t t e r done on

// the CPU as i t ’ s cons tant f o r the whole scene .

f loat3 l i gh tPos ;

l i gh tPos . x = cos (1 . 321 ∗ time 0 X ) ;

l i gh tPos . z = sin (0 . 923 ∗ time 0 X ) ;

l i gh tPos . xz = 100 ∗ normalize ( l i gh tPos . xz ) ;

l i gh tPos . y = 100 ;

// Create view vec t o r s f o r the l i g h t , l ook ing at (0 ,0 ,0)



f loat3 dirZ = −normalize ( l i gh tPos ) ;

f loat3 up = f loat3 ( 0 , 0 , 1 ) ;

f loat3 dirX = cross (up , dirZ ) ;

f loat3 dirY = cross ( dirZ , dirX ) ;

// Transform in to l i g h t ’ s view space .

// In r ea l a p p l i c a t i o n s we would be b e t t e r o f f us ing a 4x4

// matrix ins tead , but f o r t h i s shader i t ’ s cheaper to

// j u s t t ranspose and ro t a t e in to l i g h t ’ s view space .


f loat4 pos ;

In . Pos . xyz −= l i gh tPos ;

pos . x = dot ( dirX , In . Pos ) ;

pos . y = dot ( dirY , In . Pos ) ;

pos . z = dot ( dirZ , In . Pos ) ;

pos .w = 1 ;

f loat3 normalW = −normalize ( pos ) ;

// Ca l cu la t e d i f f u s e l i g h t i n g

f loat d i f f u s e = min ( 1 . 0 ,max(0 ,dot(− l i gh tD i r , normalW ) ) ) ;

d i f f u s e = d i f f u s e ∗ d i f f u s e ;


// Ca l cu la t e t e x t u r e we igh t s

f loat hatchFactor = d i f f u s e ∗ 6 . 0 ;

f loat3 weight0 = 0 . 0 ;


i f ( hatchFactor >5.0)

{weight0 . x = 1 . 0 ;

} // End i f

else i f ( hatchFactor >4.0)

{weight0 . x = 1 .0 − ( 5 . 0 − hatchFactor ) ;

weight0 . y = 1 .0 − weight0 . x ;

} // End e l s e i f


{weight0 . y = 1 .0 − ( 4 . 0 − hatchFactor ) ;

weight0 . z = 1 .0 − weight0 . y ;



{weight0 . z = 1 .0 − ( 3 . 0 − hatchFactor ) ;

weight1 . x = 1 .0 − weight0 . z ;










Out . HatchWeights0 = weight0 ;


return Out ;

}


The diffuse lighting value is calculated based on the direction of the light andthe surface normal in the diffuse variable. Then for each texture sample a weightvalue is calculated based on the diffuse value and stored in the weight parameters.The weight parameters are the output of the vertex shaders and are used in thepixel shader to generate the colors of pixels.

4.2.2.2 Pixel shader

The pixel shader takes the weighted samples from each texture and returns theirsum:

sampler Hatch0 ;

sampler Hatch1 ;

sampler Hatch2 ;

sampler Hatch3 ;

sampler Hatch4 ;

sampler Hatch5 ;

struct PS INPUT

{f loat2 TexCoord : TEXCOORD0;



} ;

struct PS OUTPUT

{f loat4 Color : COLOR0;

} ;

PS OUTPUT ps main ( PS INPUT In )

{PS OUTPUT Out ;

//Sample each t e x t u r e and weight the sample .

f loat4 hatchTex0 = tex2D(Hatch0 , In . TexCoord ) ∗ In . HatchWeights0 . x ;

f loat4 hatchTex1 = tex2D(Hatch1 , In . TexCoord ) ∗ In . HatchWeights0 . y ;

f loat4 hatchTex2 = tex2D(Hatch2 , In . TexCoord ) ∗ In . HatchWeights0 . z ;

f loat4 hatchTex3 = tex2D(Hatch3 , In . TexCoord ) ∗ In . HatchWeights1 . x ;

f loat4 hatchTex4 = tex2D(Hatch4 , In . TexCoord ) ∗ In . HatchWeights1 . y ;

f loat4 hatchTex5 = tex2D(Hatch5 , In . TexCoord ) ∗ In . HatchWeights1 . z ;

//Sum the weighted t e x t u r e va lue s

f loat4 hatchColor = hatchTex0 +

hatchTex1 +

hatchTex2 +

hatchTex3 +

hatchTex4 +

hatchTex5 ;

Out . Color = hatchColor ;

return Out ;

}


Figure 4.3: Hatching applied to platform (left), shadow hatched with differenttexture (right).

The pixel shader receives texture coordinates, and texture weights – calculatedby the vertex shader – in the PS INPUT data structure. The tex2D functionreads pixel color from textures specified by texture coordinates and a texture.To generate the hatching effect 6 hatching textures are used and a weighted sumof all the six textures results in the output color of the pixels. The result can beseen in figure 4.3 on the left.

4.2.2.3 Stream mapping

To be able to use the textures in the shader programs a texture coordinate pa-rameter has to be added to the stream map. Also for calculating diffuse lightinga surface normal is added to the map.

4.2.3 Shadow hatching

To draw shadowed areas with a different hatching pattern the shadow map have tobe taken in use. The projected shadow map values are compared to the distancebetween the platform geometry and the camera in order to decide whether apixel is in the shadow. The pixel color is masked by the shadow variable so thatshadowed pixels use the differently hatched texture.

4.2.3.1 Vertex shader

. . . // par t s exp la ined be f o re omit ted

// fo r shadow

f loat ob j e c tL ightDi s tance ;

f loat d i s t an c eS ca l e : register ( c10 ) ;

struct VS INPUT





} ;

struct VS OUTPUT

{f loat4 Pos : POSITION;

f loat3 l i ghtVec : TEXCOORD0;

f loat4 shadowCrd : TEXCOORD1;




} ;

VS OUTPUT vs main ( VS INPUT In )

{

. . . // par t s exp la ined be f o r e omit ted

// Pro jec t i t . For t h i s sample us ing the normal p ro j e c t i on

// matrix s u f f i c e s , however , r e a l a p p l i c a t i o n s w i l l t y p i c a l l y

// use a separa te p ro j e c t i on matrix f o r each l i g h t depending

// on i t s p r op e r t i e s such as FOV and range .

f loat4 sPos = mul( pro j matr ix , pos ) ;

// Use p r o j e c t i v e t e x t u r i n g to map the po s i t i on o f each fragment

// to i t s corresponding t e x e l in the shadow map.

sPos . z += 10 ;

Out . shadowCrd . x = 0 .5 ∗ ( sPos . z + sPos . x ) ;

Out . shadowCrd . y = 0 .5 ∗ ( sPos . z − sPos . y ) ;

Out . shadowCrd . z = 0 ;

Out . shadowCrd .w = sPos . z ;

ob j e c tL ightD i s tance=l i gh tPos . z−sPos . z ;

. . . // par t s exp la ined be f o r e omit ted

return Out ;

}

The shadow map is projected to the image seen and the projected shadow mapvalue is passed to the pixel shader in the shadowCrd data structure.

4.2.3.2 Pixel shader

f loat shadowBias ;

f loat shadowShadeOffset ;

sampler Hatch0 ;

sampler Hatch1 ;

sampler Hatch2 ;

sampler Hatch3 ;

sampler Hatch4 ;

sampler Hatch5 ;

sampler Hatch20 ;


sampler Hatch21 ;

sampler Hatch22 ;

sampler Hatch23 ;

sampler Hatch24 ;

sampler Hatch25 ;

sampler ShadowMap ;

struct PS OUTPUT


} ;

PS OUTPUT ps main ( f loat3 l i gh tVec : TEXCOORD0,

f loat4 shadowCrd : TEXCOORD1,

f loat2 TexCoord : TEXCOORD2,

f loat3 HatchWeights0 : TEXCOORD3,

f loat3 HatchWeights1 : TEXCOORD4

) : COLOR

{PS OUTPUT Out ;

// Radial d i s t ance

f loat depth = length ( l i ghtVec ) ;

// Normalizes l i g h t vec to r

l i ghtVec /= depth ;

// The depth o f the fragment c l o s e s t to the l i g h t

f loat shadowMap = tex2Dproj (ShadowMap , shadowCrd ) ;

// I f the depth i s l a r g e r than the s to red depth , t h i s fragment

// i s not the c l o s e s t to the l i g h t , t ha t i s we are in shadow .

// Otherwise , we ’ re l i t . Add a b i a s to avoid p r e c i s i on i s s u e s .

f loat shadow = ( depth < shadowMap + shadowBias ) ;

f loat4 hatchTex0 = tex2D(Hatch0 , TexCoord ) ∗ HatchWeights0 . x ;

f loat4 hatchTex1 = tex2D(Hatch1 , TexCoord ) ∗ HatchWeights0 . y ;

f loat4 hatchTex2 = tex2D(Hatch2 , TexCoord ) ∗ HatchWeights0 . z ;





hatchTex1 +

hatchTex2 +

hatchTex3 +

hatchTex4 +

hatchTex5 ;

f loat4 shadowHatchTex0 = tex2D( Hatch20 , TexCoord ) ∗( HatchWeights0 . x−shadowShadeOffset ) ;

f loat4 shadowHatchTex1 = tex2D( Hatch21 , TexCoord ) ∗( HatchWeights0 . y−shadowShadeOffset ) ;

f loat4 shadowHatchTex2 = tex2D( Hatch22 , TexCoord ) ∗( HatchWeights0 . z−shadowShadeOffset ) ;

f loat4 shadowHatchTex3 = tex2D( Hatch23 , TexCoord ) ∗( HatchWeights1 . x−shadowShadeOffset ) ;

f loat4 shadowHatchTex4 = tex2D( Hatch24 , TexCoord ) ∗( HatchWeights1 . y−shadowShadeOffset ) ;

f loat4 shadowHatchTex5 = tex2D( Hatch25 , TexCoord ) ∗( HatchWeights1 . z−shadowShadeOffset ) ;

f loat4 shadowHatchColor = shadowHatchTex0 +


Figure 4.4: Distance based shading of a transparent shadow texture.

shadowHatchTex1 +

shadowHatchTex2 +

shadowHatchTex3 +

shadowHatchTex4 +

shadowHatchTex5 ;

Out . Color = shadow∗hatchColor + (1−shadow )∗ shadowHatchColor ;

return Out ;

}

The pixel shader decides whether the current pixel is in the shadow. If it isnot in the shadow it uses the texture described in the previous section. If thepixel is in the shadow a different texture (with a hatching rotated 45 degrees) isused.

The result can be seen in Figure 4.3 to the right.

4.2.4 Distance based shading of a transparent shadow texture

Though the result of the previous section is already acceptable, the shader can berefined further. The textures of the shadowed areas could be mixed with the basetexture and the shadow shade would look much nicer if it would be lighter withgrowing distance from the object. The texture mixing can be achieved easily bycalculating a weighted average of the diffuse and shadow textures.

To achieve a lighter shadow shade further from the object the w coordinateof the shadowCrd parameter can be used. It holds the z value of the projectedgeometry which is an increasing value as we get away from the object. This valuecan be scaled to be used for weighting the shadow value. Based on the weightedvalue, texture weights are calculated the same way as the diffuse lighting value.The source code of the shaders can be found in the Appendix and the result inFigure 4.4.

Chapter 5

Conclusion and future work

5.1 Conclusion

The thesis set out to show that current real-time non-photorealistic renderingmethods and shadow rendering techniques can be combined to result in moreinteresting and complete images.

The present status of both fields was summarized and the development ofa possible real-time non-photorealistic rendering shader program was presented.It was shown in the thesis that NPR shadows can be rendered easily with theuse of existing methods resulting in visually more appealing shadow renditions.Current video hardware is quite capable of calculating real-time NPR shadowsopening up an infinitely various area of computer graphics. Hopefully more andmore applications will take use of this new possibility.

5.2 Future work

Besides merely combining existing NPR methods with shadow rendering possi-bilities – a huge area of possible research already – changing the shape of theshadow to apply special artistic effects algorithmically could be one extremelyinteresting area of study. Numerous films play with the idea of objects casting adifferent shape of shadow than the observer would expect to add dramatic effectto a scene. Besides the exploration of achieving this effect algorithmically forartistic purposes the idea of adding information to shadow shapes or colors couldbe generalized and also applied to other NPR fields like scientific or technicalillustrations.

51

Bibliography

[AAM03] Ulf Assarson and Tomas Akenine-Moller. A geometry-based soft shadowvolume algorithm using graphics hardware. ACM Transactions on Graphics(Proceedings of SIGGRAPH 2003), 2003. [cited at p. 29]

[Bra03] Stefan Brabec. Shadow Techniques for Interactive and Real-Time applica-tions. PhD thesis, Max-Planck-Institut fur Informatik Saarbrucken, Ger-many, 2003. [cited at p. 19, 20, 21, 23, 24, 68]

[Cav07] The chauvet-pont-d’arc cave website, (http://www.culture.gouv.fr/culture/arcnat/chauvet/en/), 2007. [cited at p. 5]

[Cha04] Eric Chan. Efficient shadow algorithms on graphics hardware. Master’sthesis, Massachusetts Institute of Technology, 2004. [cited at p. 19, 20, 21, 22,

25, 29, 68]

[Cro77] Franklin C. Crow. Shadow algorithms for computer graphics. In Proceedingsof ACM SIGGRAPH, pages 242–248. ACM Press, 1977. [cited at p. 21]

[Die01] Sim Dietrich. Shadow techniques. Nvidia Corporation, 2001. [cited at p. 20]

[FS94] Adam Finkelstein and David H. Salesin. Multiresolution curves. ComputerGraphics, 28(Annual Conference Series):261–268, 1994. [cited at p. 10]

[FvDFH96] James D. Foley, Andries van Dam, Steven K. Feiner, and John F. Hughes.Computer Graphics - Principles and Practice. The Systems ProgrammingSeries. Addison-Wesley, second edition, 1996. [cited at p. 31]

[FXC07] Nvidia’s fx composer webpage (http://developer.nvidia.com/object/fx composer home.html), 2007. [cited at p. 36]

[Gas04] Bamber Gascoigne. How to Identify Prints. Thames and Hudson, 2nd edi-tion, 2004. [cited at p. 12]

[GLY+03] Naga K. Govindaraju, Brandon Lloyd, Sung-Eui Yoon, Avneesh Sud, andDinesh Manocha. Interactive shadow generation in complex environments.In ACM Transactions on Graphics (TOG), 2003. [cited at p. 25]

53

54 BIBLIOGRAPHY

[HBS00] Wolfgang Heidrich, Stefan Brabec, and Hans-Peter Seidel. Soft shadow mapsfor linear lights high-quality. In 11th Eurographics Workshop on Rendering,pages 269–280, 2000. [cited at p. 27]

[Hei91] Tim Heidmann. Real shadows real time. In IRIS Universe, 1991.[cited at p. 23]

[HF05] Hugh Honour and John Fleming. A World History of Art. Laurence KingPublishing 7Rev Ed edition, 2005. [cited at p. 5]

[HLHS03] J.-M. Hasenfratz, M. Lapierre, N. Holzschuch, and F.X. Sillion. A survey ofreal-time soft shadows algorithms. In Eurographics 2003, 2003. [cited at p. 26,

27, 28, 68]

[KM99] Brett Keating and Nelson Max. Shadow penumbras for complex objects bydepth-dependent filtering of multi-layer depth images. pages 197–212, 1999.[cited at p. 27]

[Lai05] Samuli Laine. Split-plane shadow volumes. In Graphics Hardware, 2005.[cited at p. 24]

[McC00] Michael D. McCool. Shadow volume reconstruction from depth maps. InACM Transactions on Graphics (TOG), 2000. [cited at p. 25]

[PHWF01] Emil Praun, Hugues Hoppe, Matthew Webb, and Adam Finkelstein. Real-time hatching. In Eugene Fiume, editor, SIGGRAPH 2001, ComputerGraphics Proceedings, pages 579–584, 2001. [cited at p. 12, 68]

[REV07] Re:vision effects website (http://www.revisionfx.com/company/about/),2007. [cited at p. 13, 68]

[RMD07] Ati rendermonkey documentation, (http://ati.amd.com/developer/rendermonkey/downloads.html), 2007. [cited at p. 35]

[SL04] Sebastien St-Laurent. Shaders for Game Programmers and Artists. CourseTechnology PTR, 2004. [cited at p. 33, 39, 40, 69]

[SS98] Cyril Soler and Francois Sillion. Fast calculation of soft shadow texturesusing convolution. In Computer Graphics Proceedings, pages 321–332, Jul1998. Annual Con. [cited at p. 28]

[SS02] Thomas Strothotte and Stefan Schlechtweg. Non-Photorealistic ComputerGraphics: Modeling, Rendering and Animation. Morgan Kaufmann, 2002.[cited at p. 9, 10, 11, 12, 68]

[WFH+97] Daniel N. Wood, Adam Finkelstein, John F. Hughes, Craig E. Thayer, andDavid H. Salesin. Multiperspective panoramas for cel animation. ComputerGraphics, 31(Annual Conference Series):243–250, 1997. [cited at p. 8]

[Wil78] Lance Williams. Casting curved shadows on curved surfaces. In Proceedingsof ACM SIGGRAPH, pages 270–274. ACM Press, 1978. [cited at p. 18]

[YTD02] Zhengming Ying, Min Tang, and Jinxiang Dong. Soft shadow maps forarea light by area approximation. In 10th Pacific Conference on ComputerGraphics and Applications, pages 442–443. IEEE, 2002. [cited at p. 27]

55

[Zan98] Hansong Zang. Forward shadow mapping. In Eurographics Rendering Work-shop 1998, pages 131–138, 1998. [cited at p. 18]

Appendices

57

Appendix A

NPR shadow pass vertex

shader source code

f loat4 l i g h tD i r ;

f loat ob j e c tL ightD i s tance ;

float4x4 view matr ix ;

float4x4 v i ew pro j mat r ix : register ( c0 ) ;

f loat4 l i gh tPos : register ( c4 ) ;

f loat4 v i ew po s i t i on : register ( c5 ) ;

float4x4 pro j mat r ix : register ( c6 ) ;

f loat d i s t an c eS ca l e : register ( c10 ) ;

f loat time 0 X : register ( c11 ) ;

struct VS OUTPUT {f loat4 Pos : POSITION;

f loat3 l i ghtVec : TEXCOORD0;

f loat4 shadowCrd : TEXCOORD1;




f loat3 ShadowHatchWeights0 :TEXCOORD5;

f loat3 ShadowHatchWeights1 :TEXCOORD6;

} ;

struct VS INPUT




} ;

VS OUTPUT main (VS INPUT In ){VS OUTPUT Out ;

// Animate the l i g h t po s i t i on .

// Comment out t h i s code to use a s t a t i c l i g h t .

59

60 APPENDIX A. NPR SHADOW PASS VERTEX SHADER SOURCE CODE



f loat3 l i gh tPos ;

l i gh tPos . x = cos (1 . 321 ∗ time 0 X ) ;

l i gh tPos . z = sin (0 . 923 ∗ time 0 X ) ;

l i gh tPos . xz = 100 ∗ normalize ( l i gh tPos . xz ) ;

l i gh tPos . y = 100 ;

// Flip , s c a l e and t r a n s l a t e our model to s u i t our scene

// In r ea l app l i c a t i on s , t h i s work shou ld normal ly be

// done at load time , a l t e r n a t i v e l y i f animation i s des i red ,

// be a l t e r e d by a world matrix .

In . Pos . xyz = 1 .6 ∗ In . Pos . xyz ;

In . Pos . y −= 40 ;

In . Normal = In . Normal . xyz ;

Out . Pos = mul( v i ew pro j matr ix , In . Pos ) ;

// World−space l i g h t i n g

Out . l i ghtVec = d i s t an c eS ca l e ∗ ( l i gh tPos − In . Pos . xyz ) ;

// Create view vec t o r s f o r the l i g h t , l ook ing at (0 ,0 ,0)



f loat3 dirZ = −normalize ( l i gh tPos ) ;

f loat3 up = f loat3 ( 0 , 0 , 1 ) ;

f loat3 dirX = cross (up , dirZ ) ;

f loat3 dirY = cross ( dirZ , dirX ) ;

// Transform in to l i g h t ’ s view space .

// In r ea l a p p l i c a t i o n s we would be b e t t e r o f f us ing a 4x4

// matrix ins tead , but f o r t h i s shader i t ’ s cheaper to

// j u s t t ranspose and ro t a t e in to l i g h t ’ s view space .

f loat4 pos ;

In . Pos . xyz −= l i gh tPos ;

pos . x = dot ( dirX , In . Pos ) ;

pos . y = dot ( dirY , In . Pos ) ;

pos . z = dot ( dirZ , In . Pos ) ;

pos .w = 1 ;

// Pro jec t i t . For t h i s sample us ing the normal p ro j e c t i on

// matrix s u f f i c e s , however , r e a l a p p l i c a t i o n s w i l l t y p i c a l l y

// use a separa te p ro j e c t i on matrix f o r each l i g h t depending

// on i t s p r op e r t i e s such as FOV and range .

f loat4 sPos = mul( pro j matr ix , pos ) ;

// Use p r o j e c t i v e t e x t u r i n g to map the po s i t i on o f each fragment

// to i t s corresponding t e x e l in the shadow map.

sPos . z += 10 ;

Out . shadowCrd . x = 0 .5 ∗ ( sPos . z + sPos . x ) ;

Out . shadowCrd . y = 0 .5 ∗ ( sPos . z − sPos . y ) ;

Out . shadowCrd . z = 0 ;

Out . shadowCrd .w = sPos . z ;

ob j e c tL ightD i s tance=l i gh tPos . z−sPos . z ;

Out . TexCoord = In . TexCoord ;

f loat3 normalW = −normalize ( pos ) ;

f loat d i f f u s e = min ( 1 . 0 ,max(0 ,dot(− l i gh tD i r , normalW ) ) ) ;

61



f loat hatchFactor = d i f f u s e ∗ 6 . 0 ;




{weight0 . x = 1 . 0 ;

} // End i f























Out . ShadowHatchWeights0 = weight0 ;

Out . ShadowHatchWeights1 = weight1 ;

return Out ;

}

Appendix B

NPR shadow pass pixel shader

source code

f loat backProject ionCut ;

f loat shadowBias ;

f loat ob j e c tL ightD i s tance ;

sampler ShadowMap ;

f loat shadowDistM ;

f loat shadowDistD ;

f loat shadowShadeOffset ;

sampler Hatch0 ;

sampler Hatch1 ;

sampler Hatch2 ;

sampler Hatch3 ;

sampler Hatch4 ;

sampler Hatch5 ;

sampler Hatch20 ;

sampler Hatch21 ;

sampler Hatch22 ;

sampler Hatch23 ;

sampler Hatch24 ;

sampler Hatch25 ;

struct PS OUTPUT


} ;

PS OUTPUT main (

f loat3 l i gh tVec : TEXCOORD0,

f loat4 shadowCrd : TEXCOORD1,

f loat2 TexCoord : TEXCOORD2,


63

64 APPENDIX B. NPR SHADOW PASS PIXEL SHADER SOURCE CODE


f loat3 ShadowHatchWeights0 : TEXCOORD5,

f loat3 ShadowHatchWeights1 : TEXCOORD6

) : COLOR

{PS OUTPUT Out ;

// Radial d i s t ance

f loat depth = length ( l i ghtVec ) ;

// Normalizes l i g h t vec to r

l i ghtVec /= depth ;

// The depth o f the fragment c l o s e s t to the l i g h t

f loat shadowMap = tex2Dproj (ShadowMap , shadowCrd ) ;

// I f the depth i s l a r g e r than the s to red depth , t h i s fragment

// i s not the c l o s e s t to the l i g h t , t ha t i s we are in shadow .

// Otherwise , we ’ re l i t . Add a b i a s to avoid p r e c i s i on i s s u e s .

f loat shadow = ( depth < shadowMap + shadowBias ) ;

// Cut back−pro j ec t i on , t ha t i s , make sure we don ’ t l i t

// anything behind the l i g h t . Theore t i ca l l y , you shou ld j u s t

// cut at w = 0 , but in p ra c t i c e you ’ l l have to cut at a

// f a i r l y h igh p o s i t i v e number to avoid p r e c i s i on i s s u e when

// coord ina te s approaches zero .

shadow ∗= ( shadowCrd .w > backProject ionCut ) ;








hatchTex1 +

hatchTex2 +

hatchTex3 +

hatchTex4 +

hatchTex5 ;

f loat hatchFactor ;

hatchFactor = 6∗( backProject ionCut /( shadowCrd .w−ob j e c tL ightD i s tance ) ) ;




{weight0 . x = 1 . 0 ;

} // End i f





65

















f loat4 shatchTex0 = tex2D( Hatch20 , TexCoord ) ∗ weight0 . x ;

f loat4 shatchTex1 = tex2D( Hatch21 , TexCoord ) ∗ weight0 . y ;

f loat4 shatchTex2 = tex2D( Hatch22 , TexCoord ) ∗ weight0 . z ;

f loat4 shatchTex3 = tex2D( Hatch23 , TexCoord ) ∗ weight1 . x ;

f loat4 shatchTex4 = tex2D( Hatch24 , TexCoord ) ∗ weight1 . y ;

f loat4 shatchTex5 = tex2D( Hatch25 , TexCoord ) ∗ weight1 . z ;

f loat4 shatchColor = shatchTex0 +

shatchTex1 +

shatchTex2 +

shatchTex3 +

shatchTex4 +

shatchTex5 ;

Out . Color = shadow∗hatchColor + (1−shadow )∗ (4∗ shatchColor/5+hatchColor / 5 ) ;

return Out ;

}

List of Symbols

and Abbreviations

Abbreviation Description Mentioned

NPR Non-Photorealistic Rendering page 7API Application Programming Interface page 31CAD Computer Aided Design page 14IDE Integrated Development Environment page 34ISO International Organisation for Standardization page 15GPU Graphics Processing Unit page 31DirextX DirectX is a collection of APIs for handling mul-

timedia tasks.page 31

OpenGL Open Graphics Library page 31HLSL High Level Shader Language page 33Cg C for Graphics is a high-level shading language

created by NVIDIA for programming vertex andpixel shaders

page 33

GLSL OpenGL Shading Language page 33ATI ATI Technologies U.L.C. is a major designer of

graphics processing units and video display cardsand a wholly owned subsidiary of AMD

page 35

NVIDIA NVIDIA Corporation is a major supplier of graph-ics processors, graphics cards, media and commu-nications devices for PCs and game consoles

page 33

67

List of Figures

1.1 Artistic screening with non-linear screen element mapping [SS02] . . . 9

1.2 Objects shaded with real-time hatching effect. [PHWF01] . . . . . . . 12

1.3 A frame from the Oscar winning film for best visual effects usingpainterly style: ”What Dreams May come” [REV07] (Copyright 1998PolyGram Filmed Entertainment Distribution, Inc.) . . . . . . . . . . 13

1.4 Technical illustration with ghosting view (Copyright 1994-2006 KevinHulsey Illustration) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

2.1 Geometric configuration of the shadow map algorithm. Sample A isconsidered to be in shadow because the point on the receiving planeis farther from the light source than the distance stored in the shadowmap (which belongs to the blocker). In contrast, sample B is illumi-nated. [Cha04] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

2.2 Bias related artifacts. (a.) Too little bias: incorrect self shadowingeffect - pixels falsely determined to be in shadow (b.) Too much biasapplied: shadow boundary problem. (c.) Correct bias. [Cha04] . . . . 20

2.3 Shadow map aliasing: (a.) 256x256, (b.) 512x512, (c.) 1024x1024shadow maps. [Cha04] . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

2.4 Inside-outside test: Counting intersections with shadow volumes (in-crementing at front facing polygons and decrementing at back facingpolygons) give zero for non-shadowed pixels and non-zero for shad-owed pixels. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

2.5 Shadow volumes: Final image (left), Silhouette edges (middle), Shadowvolumes (right) [Bra03] . . . . . . . . . . . . . . . . . . . . . . . . . . 23

2.6 Soft shadows [HLHS03] . . . . . . . . . . . . . . . . . . . . . . . . . . 26

2.7 Combining several hard shadows. Left: 4 samples. Right: 1024 sam-ples [HLHS03] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

2.8 Inner and outer penumbra calculated from point light. [HLHS03] . . 28

68

69

3.1 A possible hardware rendering architecture including vertex and pixelshaders. [SL04] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

3.2 Screenshot of ATI’s Rendermonkey shader development environment. 35

4.1 Screenshots of the reused Rendermonkey example shaders NPR/Hatch(left) and Shadows/Shadowmapping (right). . . . . . . . . . . . . . . . 40

4.2 Disc object intersecting the trefoil knot object (left), fixed disc scaleand position (right). . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

4.3 Hatching applied to platform (left), shadow hatched with differenttexture (right). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

4.4 Distance based shading of a transparent shadow texture. . . . . . . . . 49

real-time non-photorealistic shadow rendering

Education

npr shadow rendering

npr shadows

shadow maps

shadow hatching46

shadow concepts

npr possibilities

npr techniques10

stencil shadow volumes