wilf lalonde ©2012 comp 4501 95.4501 water. wilf lalonde ©2012 comp 4501 four properties that...

$: Wilf LaLonde ©2012 Comp 4501 95.4501 Water. Wilf LaLonde ©2012 Comp 4501 Four properties that engines either control or fake... Waves Reflection Refraction$
Wilf LaLonde ©2012Comp 4501

95.450195.4501

Water


• Four properties that engines either control or fake...

• Waves

• Reflection

• Refraction

• Flow

WaterWater


• Environment mapping via static cube maps are often used to provide simple reflection and refraction without needing to draw the world in multiple passes.

• Dynamic cube maps are also used to provide more locally correct environment mapping.

• Multiple world drawing passes and frame buffers can be used instead of cube maps to provide more accurate results.

• Wave simulation provides even better results.

TechniquesTechniques


95.450195.4501

Environment

Mapping + Cube

Maps


• Environment mapping is a technique that encodes the environment in a map (most simply a cube map); also called a skybox.

• Works well under the assumption that the environment is infinitely far.

• Typical applications

• Skybox (can never get to the box).

• Vehicle shine (give it a chrome appearance); works best on curved surfaces.

• Water shine to reflect fuzzy sky.

Environment MappingEnvironment Mapping


Cube MapsCube Maps

L F R B

T

BHumus viewer


• Via pictures taken by camera and stitched with special software.

• Drawn or assembled by artists.• Built by the game engine by taking 6 pictures

with 90 degrees field of view and square aspect ratio...

Halflife worlds are filled with boxes that say “make static cube map here” or “make dynamic cube map when near here”

Building Cube MapsBuilding Cube Maps


• Cube maps are sampled with an unnormalized 3D direction vector (the pixels are conceptually at infinity).

e.g., texCUBE (sample, direction)

• The primitive deals with making the corners LOOK smoothly rounded...

Sampling Cube MapsSampling Cube Maps

So don’t need to bother normalizing vector if sampling cube map


• Reflection/refraction are built in shader primitives...

Reflection/Refraction Shader PrimitivesReflection/Refraction Shader Primitives

NIn R

R = reflect (In, N)For N normalized is mirror reflection

N1

2

Snell's Law1 sin 1 = 2 sin 2

h = index of refractionAir (1), Water (1.33), Glass

(1.5)

1

2

R = refract (In, N, r)For N normalized and r is ratio 2 / 1

R

In


• The fresnel effect dictates the proportion of reflection to refraction via a reflection coefficient t (In is incident ray, N is normal)

t = clamp01 (bias + scale * (1 + In.N)power))

• So you can interpolate via

lerp (refraction, reflection, t)

Simultaneous Reflection/RefractionSimultaneous Reflection/Refraction

shiny car: bias=0.05 scale=0.95 power=3.0glass: bias=0.04 scale=1.0 power=2.0water: bias=0.6 scale=0.8 power=6.0

When looking straight down, I is aligned against N (to give -1), so with no bias, t = 0 for no reflectionj.


• Chromatic dispersion dictates that red, green, blue components have decreasing index of refraction ratios; e.g., 1.33, 1.27, 1.2 resp. for water.

Chromatic DispersionChromatic Dispersion

N1

2

1

2

red

I

greenblue


float fresnelRatio (float bias, float scale, float power, float3 incidentRay, float3 normal) {return clamp01 (bias + scale * pow (1.0 + dot (incidentRay, normal), power));

}

float waterFresnelRatio (float3 incidentRay, float3 normal) {return fresnelRatio (0.6, 0.8, 6.0, incidentRay, normal);

}

float3 refract (float3 incidentRay, float3normal, float3 indexOfRefractionRatio) {//This is chromatic refract. The compiler will choose this “refract” method if a//float3 index of refraction ratio is provided instead of a single float...return float3 (

refract (incidentRay, normal, indexOfRefractionRatio .x),refract (incidentRay, normal, indexOfRefractionRatio .y),refract (incidentRay, normal, indexOfRefractionRatio .z));

}

Shader FunctionsShader Functions

shiny car: bias=0.05 scale=0.95 power=3.0glass: bias=0.04 scale=1.0 power=2.0water: bias=0.6 scale=0.8 power=6.0


• Skyboxes are inward facing cubes with 6 pictures of very far objects that surround you (left/right, top/down, front/back); e.g., mountains.

• They can be drawn in 2 ways (a face is a quad).

• By drawing 6 faces of a large cube taking special care that the pixels at the borders touch at the mid point (pixel perfect drawing).

• By drawing 1 face via 1 cube texture that contains the 6 2D texture in a special way.

SkyboxesSkyboxes


Cube Textures or MapsCube Textures or Maps

L F R B

T

B

back

bottom


• glTexImage2D (GL_TEXTURE_2D, 0,GL_RGBA8, width, height, 0, GL_RGBA, GL_UNSIGNED_BYTE ,

data);• glTexImage2D (GL_TEXTURE_2D, 0,

GL_RGB8, width, height, 0, GL_RGB, GL_UNSIGNED_BYTE , data);

• glTexImage2D (GL_TEXTURE_CUBE_MAP_POSITIVE_X , 0,0, GL_RGBA8, width, height, 0, GL_RGBA, GL_UNSIGNED_BYTE,

data);

Loading textures in OpenGLLoading textures in OpenGL

internal format external format

GL_TEXTURE_CUBE_MAP_POSITIVE_X

GL_TEXTURE_CUBE_MAP_POSITIVE_Y

GL_TEXTURE_CUBE_MAP_POSITIVE_Z

GL_TEXTURE_CUBE_MAP_NEGATIVE_X

GL_TEXTURE_CUBE_MAP_NEGATIVE_Y

GL_TEXTURE_CUBE_MAP_NEGATIVE_Z

cube map has 6 pieces (so do this 6 times)

• GLboolean mipmap = GL_TRUE); //or GL_FALSEglTexParameteri (GL_TEXTURE_2D, GL_GENERATE_MIPMAP, mipmap);

Older OpenGL style uses extra call

gluBuild2DMipmapsafter

glTexImage2D(with similar parameters)

if you want mipmaps


void Renderer::DrawSkybox () {

glDepthMask (GL_FALSE);glEnable (GL_TEXTURE_CUBE_MAP_SEAMLESS );

SetCurrentShader (skyboxShader);UpdateShaderMatrices ();quad -> Draw (); //At [0,0,0]; It’s units are -0.5

to +0.5glUseProgram (0);

glDisable (GL_TEXTURE_CUBE_MAP_SEAMLESS );glDepthMask (GL_TRUE);

}

Pseudo Code for Drawing SkyboxPseudo Code for Drawing Skybox

But you need to do something special in the SHADERS to make it work...


• Note that if the camera is at an arbitrary spot in the world, drawing a quad at [0,0,0] in the standard way could draw nothing (e.g., if [0,0,0] is behind the camera).

• Issue 1: We need to make sure the quad is on the screen and NOT clipped by the camera’s “NEAR” plane.

• Issue 2: We also need to make sure that even with a wide field of view (e.g, 100 degrees instead of a more traditional 60 degrees), we can’t see any part OUTSIDE the quad. So we may need to scale the quad...

Drawing the SkyboxDrawing the Skybox


• Claim 1: If F were the matrix for a flying bird, and we want to see from the bird’s point of view, the view matrix would be F-1.

• In our case, we’ll use C instead of F to denote the camera. Hence the view matrix would be C-1.

Step 1: Understanding How To Draw The Cube Map With a QuadStep 1: Understanding How To Draw The Cube Map With a Quad

Why? Explained on the next slide...


• Experiment: Suppose we have a point [-1,0.0] on our left and the camera is at the origin facing straight ahead.

• So clearly, we can’t see it... Let’s change C to the matrix ROTATE LEFT (by 90 degrees). So the view matrix V changes in such a way that the point in view space is [0,0, -1]; i.e., one meter in front of us. What’s V?

Step 1: Understanding How To Draw The Cube Map With a QuadStep 1: Understanding How To Draw The Cube Map With a Quad

Camera C = Identity (at [0,0,0] looking straight ahead)

[0,0,0][-1,0,0]


• So far, [-1,0,0] in world space is [0,0,-1] in view space.

• What’s V? Clearly, [-1,0, 0] * V = [0,0, -1]. What did V do to [-1,0, 0]?

If C is the camera matrix, the view matrix V = C-1If C is the camera matrix, the view matrix V = C-1

[0,0,0][-1,0,0]

[0,0,0][-1,0,0]

[0,0,-1]

If C = ROTATE LEFT, V = ROTATE RIGHT

V rotated [-1,0,0] RIGHT to produce [0,0,-1]


• Although V = C-1, let’s not use V to draw the quad at [0,0,0]... What if we use I (the identity) instead.

• It will get clipped by the near clipping plane... Typical values for near plane distance is 0.01, 0.1, 0.2 (let’s assume 1 is way past the clipping plane).

So what do we Do?So what do we Do?

[0,0,0]

Quad at [0,0,0] with units in range -0.5 to +0.5

Near plane

Far plane (far away)


• If we move it ahead by one, it will NOT completely cover the screen if a wide-angle field of view is used...

• So scale the quad by a huge amount; e.g., 10. The draw routine will clip it to the screen.

• What we need to do. If p is a point on the quad,

transform it by p * 10 - 1

So what do we Do?So what do we Do?

[0,0,0]

Normal FOV

Near plane

Far plane (far away)

Wide angle FOV

scale by 10, move ahead by 1


• We want to be able to take a point p on the screen and convert it to a uv coordinate (for cube maps, we need 3D directions, not points).

• Note that “p - [0,0,0]” is a vector. But is it the correct one... To figure out what we need, lets consider the vector for the point right in front of us... d = [0, 0, -1].

• Recall that our camera was ROTATED LEFT. So the d we want is [-1,0,0]. How do we get it?

What uv do we use to index into the cube map?What uv do we use to index into the cube map?

[0,0,0]

Near plane

[0,0,-1]


• Recall that our camera was ROTATED LEFT. So the d we want is [-1,0,0]. How do we get it from [0,0,-1]?

• All we need is d * C. Use V3x3 * d instead...

What uv do we use to index into the cube map?What uv do we use to index into the cube map?

Vectors only need the 3x3 part of the matrix (no translation)

Near plane

[0,0,-1]

A right multiply is a multiply by “transpose (V3x3)” WHICH IS its INVERSE of V3x3 if there is no scale...

But Cameras don’t have scale


• A point p is transformed via

p * 10 - 1

• A point p is reinterpreted as a uvxyz direction and transformed via

V3x3 * p

SummarySummary

scale by 10, move ahead by 1

This is p * C BUT USING V which is C-1.


Vertex Shader For Drawing a Skybox in OpenGL...Vertex Shader For Drawing a Skybox in OpenGL...

# version 150 core//Transform position via “* 10 - 1” and direction via “* inverse (viewMatrix)”.uniform mat4 viewMatrix ;uniform mat4 projectionMatrix ;

struct INPUT {vec3 position;

};struct OUTPUT {

vec3 uvw;};INPUT input; OUTPUT output;

void main () {vec3 position = input.position * 10.0 - vec3 (0 ,0 ,1);vec3 direction = input.position;output.uvw = direction * mat3 (viewMatrix); //direction * inverse

(viewMatrix)gl_Position = projectionMatrix * vec4 (position, 1.0);

}

Warning: OpenGL Conventions are flipped. M * p is the standard left multiply

p * M is the right multiply (left multiply by transpose)


Pixel Shader For Drawing a Skybox in OpenGL...Pixel Shader For Drawing a Skybox in OpenGL...

# version 150 core//Sample the cube map with the UNNORMALIZED uvw vector.uniform samplerCube cubeTexture;

struct INPUT {vec3 uvw;

};

struct OUTPUT {vec4 color;

};

INPUT input; OUTPUT output;

void main ( void ) {

output.color = texture (cubeTexture , input.uvw);}

Warning: OpenGL Conventions are flipped. M * p is the standard left multiply

p * M is the rightmultiply (left multiply by transpose)


• Assume the game world draws a quad for a lake or ocean that is scaled and positioned in the world... via a modelMatrix.

• The following texture is supplied.

• The vertex shader provides 3D world space “position” (by multiplying by modelMatrix) and 2D “textureCoordinate” (by passing it through).

Drawing a large flat water body Reflecting The SkyboxDrawing a large flat water body Reflecting The Skybox


Pixel Shader For Drawing Skybox Reflecting WaterPixel Shader For Drawing Skybox Reflecting Water# version 150 core//Sample the cube map with the UNNORMALIZED uvw vector. uniform sampler2D waterTexture;uniform samplerCube cubeTexture;uniform vec3 cameraWorldPosition;

struct INPUT {vec4 worldPosition;vec3 textureCoordinate;

};

INPUT input; OUTPUT output;

void main ( void ) {vec3 toWater = normalize (input.worldPosition.xyz -

cameraWorldPosition);vec3 waterNormal = vec3 (0.0, 1.0, 0.0); //Up...vec3 waterReflection = reflect (towater, waterNormal);vec4 skyboxColor = texture (cubeTexture, waterReflection); vec4 waterColor = texture (waterTexture, input. textureCoordinate;const bool applyFresnel = true; //or false to disable...float reflectionT = applyFresnel

? fresnelRatio (0.6, 0.8, 6.0, toWater, normal) //bias, scale, power: 0.3;

output.color = lerp (waterColor, skyBoxColor, reflectionT);}

Use fresnel t to interpolate between water color and skybox color

struct OUTPUT {vec4 color;

};

But it won’t reflect terrain or buildingsnor will the water

move...


95.450195.4501

Flat Jiggling Water


• AuthorsHelena DuchaussoyFabien KapalaFranck LetellierBaptiste Malaga

• Where IMAC engineering school. Marne-la-Vallée, Paris, FRANCE

• WhatProgramming course

Ruins Island DemoRuins Island Demo

Demo


OpenGL uses a right to left system... So if you are used to writing a * A * B or a * (A * B) to transform a to space A and then further transform the result to space B where A and B are row based matrices with the bottom row denoting a translation, everything will seem flipped to you.

Technically, you have to write B' * A' * a or (B' * A') * a where A' is built by saying "here are the 4 columns" of A when in fact "they are actually your rows". They will get stored in such a way that everything will work from right to left as they intend...We can prove that but we would need a short discussion...

Although a point p can be tranformed into view space via viewMatrix * p, it's a little inaccurate for vectors IF the modelMatrix has non-uniform scaling such as [1, 2, 0,5]... The more correct matrix to use on normals and vectors is actually the transpose of the inverse of the viewMatrix which is equal to viewMatrix if the scale is [1,1,1]. OpenGL calls this matrix "gl_NormalMatrix"; i.e., the viewMatrix for normals...

A Note About OpenGL MatricesA Note About OpenGL Matrices


• For reflection, consider the water as a mirror... Can you draw with a reflected camera?

Mirrors For The Game EngineMirrors For The Game Engine

camera

reflected cameramirror

DO YOU SEE RED ON YOUR LEFT OR YOUR RIGHT?CAN YOU SEE BLUE?


• Do not touch the camera...• Reflect the world about the mirror...• Clip away the objects that used to be on the

back side of the mirror...

How To Mirror ProperlyHow To Mirror Properly

cameraReflect the world about this PLANE of the mirror

causes winding order to be backward

(see next slide)

requires clippling plane


• To deal with reflection, the water is considered to be a mirror... SO A TECHNIQUE IS NEEDED TO PERFORM THE REFLECTION which makes front faces back faces (see next slide).

• Also

2 Issued For The Game Engine2 Issued For The Game Engine

camera

Reflected camera


A Side-Effect of the “Reflect” TransformationA Side-Effect of the “Reflect” Transformation

• A front-facing face becomes back facing (and vice-versa).

front

This must be counteracted via command“if front face winding order is counter clockwise, make then

clockwise else make then counter clockwise”

This must be counteracted via command“if front face winding order is counter clockwise, make then

clockwise else make then counter clockwise”

1

4

2

3

1

4

2

3

tnorf

Will this be afront or back

face?


• A clipping plane pointing in the direction of normal N at a point P looks like

[Nx,Ny,Nz,-P.N]• A point Q is on the positive side of the plane if

N.Q > 0• To transform a plane by M, you actually need

to transform the 4D point by (M-1)t.

Clipping PlanesClipping Planes

a 4D point

Clipping planes are a problem for shaders because YOU HAVE TO ADD THE TEST TO ALL PIXEL SHADERS YOU DRAW WITH

“if fails positive test, discard pixel”


• Lengyel wrote a paper describing how to modify the standard built-in clipping plane that is automatically part of the perspective matrix so it clips with your plane instead of front plane.

Careful: OpenGL versus DirectX use slightly different math and Lengyel derives it for a right to left system.

I personally was not able to make it work in general (only in simple cases)

There is a trick to AVOIDING clipping planes in ShaderThere is a trick to AVOIDING clipping planes in Shader


• What the demo from France does...Creates a reflection matrix of the form “scale by [1,-1,1]”; i.e. flip y.

Uses the Lengyel approach to clipping which if you look at in the code appears to be random mutilations of the perspective matrix.

Ruins Island DemoRuins Island Demo

Works only for horizontal water at [0,0,0]

No attempt to abstract it into a nice routine or to explain it


• The water (see WILF NOTE NORMAL) is a 1X1 VERTICAL quad with width and height 400 meters built via“water = new Plan (1,400,400);”.

The Water QuadThe Water Quad

How to rotate around an arbitrary axis1. Grab the axis with the right hand (thumb

pointing in the axis direction)

2. Positive rotation is in the direction you CLOSE YOUR HAND.model space

world space

How do you rotate it so it’s horizontal and it’s normal is UP?


• The water (see WILF NOTE NORMAL) is a 1X1 quad with width and height 400 meters built as follows “water = new Plan (1,400,400);”.

The Water QuadThe Water Quad

Use -90 rotation around x-axis

-90

+90 rotation around negative x-axis

OR

model space

world space


• T, B, N are supposed to indicate where the tangent space x-/y-/z- axes are IN MODEL SPACE...

Tangent SpaceTangent Space

T = [1,0,0]B = [0,1,0]N = [0,0,1]

Search “WILF NOTE NORMAL” to see what they are using.

-90

THIS is what it looks like in model space

TBNMS = mat3 (T,B,N) maps from tangent space to model spaceTBNCS = mat3 (T,B,N) * ModelViewMatrix maps to camera space


gl_Vertex – gl_Vertex is in model space.gl_MultiTexCoord0.xy – the texture coordinate.gl_LightSource [0].position.xyz – lights in

OpenGL are always stored in camera space.

uniform float timer; //Time in millisecond#define time timer

Inputs to the Vertex ShaderInputs to the Vertex Shader

Shader version #120 (so no structs allowed)


varying vec3 toLightTS;varying vec3 toCameraTS;varying vec2 flowTextureCoordinate1; //NOT USEDvarying vec2 flowTextureCoordinate2; //NOT USEDvarying vec2 textureCoordinate;varying vec4 positionPS;

Outputs from the Vertex ShaderOutputs from the Vertex Shader


//Compute tbnCS...

//Compute toLightTS... //Compute toCameraTS... //Pass the texture coordinates through...

//Transform the pixel position to projection space.

The Vertex Shader “water.vert”The Vertex Shader “water.vert”


//Compute tbnCS...//Compute the TBN matrix to convert from tangent space to view space. We'll want

the //inverse meaning we'll uses it via vector * TBN (which operates on the transpose) //rather than TBN * vector (which operates on it directly).//Note that the transpose of a 3x3 matrix without scale is its inverse...

//Let TBN represent uvw axes in model space... Recall the water model is a quad built on a

//vertical plane with the normal coming into your eye, //we need T, B, N vectors saying where u,v,w are in model space...vec3 T = vec3 (1.0, 0.0, 0.0); //Let u be the x coordinate (right).vec3 B = vec3 (0.0, 1.0, 0.0); //Let v be the y coordinate (up).vec3 N = vec3 (0.0, 0.0, 1.0); //Let w be the z coordinate (back).

mat3 modelViewMatrix = gl_NormalMatrix; //This maps from model space to view space

//but is more accurate for normals...

//Note: mat3 tbnMS = mat3 (T,B,N); mat3 tbnCS = tbnMS * modelViewMatrix; //But OpenGL is right to left...mat3 tbnCS = modelViewMatrix * mat3 (T,B,N); //Maps from tangent to camera

space...



//Compute toLightTS...//Built-in OpenGL lights are given in view (also called camera space) positions...vec3 pixelPositionCS = vec3 (gl_ModelViewMatrix * gl_Vertex); //gl_Vertex is in model

space.vec3 lightPositionCS = gl_LightSource[0].position.xyz;vec3 toLightCS = pixelPositionCS - lightPositionCS;SHADER_OUTPUT toLightTS = toLightCS * tbnCS; //Premultiply for inverse (tbnCS) *

toLightCS;

//Compute toCameraTS...vec3 toCameraCS = /* [0,0,0] */ - pixelPositionCS;SHADER_OUTPUT toCameraTS = toCameraCS * tbnCS; //Premultiply for inverse (tbnCS) *

toCameraCS;

//Pass the texture coordinates through...SHADER_OUTPUT textureCoordinate = gl_MultiTexCoord0.xy;

//Transform the pixel position to projection space.SHADER_OUTPUT positionPS = gl_ModelViewProjectionMatrix * gl_Vertex;gl_Position = SHADER_OUTPUT positionPS;


#define SHADER_OUTPUT


varying vec3 toLightTS;varying vec3 toCameraTS;varying vec2 textureCoordinate;varying vec4 positionPS;

#define SHADER_INPUT

//Experiments#define drawOriginal 0#define drawWilfOpaquepPurple 1#define drawWilfTransparentPurple 2#define drawWilfCycleTransparentWaterColors 3#define drawWilfCycleWaterRefractionReflectionPerlin 4#define drawWilfCycleWaterReflectionRefractionPerlin1Perlin2 5#define drawWilfStraightReflect 6#define drawWilfStraightReflectButLessIfStraightDown 7#define drawWilfJiggleReflection1 8#define drawWilfJiggleReflection2 9#define drawWilfColorize 10

//#define experiment drawOriginal

The Pixel Shader “water.frag”The Pixel Shader “water.frag”


varying vec3 toLightTS;varying vec3 toCameraTS;varying vec2 textureCoordinate;varying vec4 positionPS;

#define SHADER_INPUT

//Experiments#define drawOriginal 0#define drawWilfOpaquePurple 1#define drawWilfTransparentPurple 2#define drawWilfCycleTransparentWaterColors 3#define drawWilfCycleWaterRefractionReflectionPerlin 4#define drawWilfCycleWaterReflectionRefractionPerlin1Perlin2 5#define drawWilfStraightReflect 6#define drawWilfStraightReflectButLessIfStraightDown 7#define drawWilfJiggleReflection1 8#define drawWilfJiggleReflection2 9#define drawWilfColorize 10

//#define experiment drawOriginal

The Pixel Shader “water.frag”The Pixel Shader “water.frag”


Original ShaderOriginal Shader


First 3 ExperimentsFirst 3 Experiments

drawWilfOpaquePurple drawWilfTransparentPurple

(also one of drawWilfCycleTransparentWaterColors)


Experiment drawWilfCycleWaterRefractionReflectionPerlinExperiment drawWilfCycleWaterRefractionReflectionPerlin

refraction does not work in original demo

refraction (not used)

Refraction (used)

perlin (used for surface waves)


Experiment drawWilfCycleWaterReflectionRefractionPerlin1Perlin2 Experiment drawWilfCycleWaterReflectionRefractionPerlin1Perlin2

Drawn in the final back buffer


• Draw the world with reflection into the reflection texture.

• Draw the world normallyinto the final frame buffer.

• Draw the water using perlin noise and allow some of the final frame buffer to showthrough when lookingstraight down.

The Algorithm UsedThe Algorithm Used

Both of the above DO NOT draw the water.

But they can’t jiggle it.


• It needs to be in a texture when we draw the water into the final frame buffer... We coopted one of their post-effect frame buffers “fboColorDepth” and executed

To Be Able To Jiggle The BottomTo Be Able To Jiggle The Bottom

//Copy "finalFBO"'s COLOR texture to the "fboColorDepth" framebuffer //for use by the water shader...

glActiveTexture (GL_TEXTURE0); finalFBO->activateTexture ();

ShaderManager &shaderManager = ShaderManager::getInstance ();

shaderManager.getShader ("wilfCopy")->Activate ();fboColorDepth->activate ();

glClearColor(1, 1, 1, 1);glColorMask (GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE);glClear (GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);displayOnQuad (iWindowWidth, iWindowHeight);

fboColorDepth->desactivate ();shaderManager.getShader("wilfCopy")->Desactivate ();

finalFBO->desactivateTexture();


The WilfCopy ShadersThe WilfCopy Shaders

THE VERTEX SHADER

varying vec2 textureCoordinate;

void main () {textureCoordinate = gl_MultiTexCoord0.xy;gl_Position = ftransform();

}

THE PIXEL SHADER

uniform sampler2D texture;varying vec2 textureCoordinate;

void main () {gl_FragColor = texture2D (texture, textureCoordinate);

}


Experiment drawWilfStraightReflectExperiment drawWilfStraightReflect


Experiment drawWilfStraightReflectButLessIfStraightDownExperiment drawWilfStraightReflectButLessIfStraightDown


Experiment drawWilfJiggleReflection1 AND 2Experiment drawWilfJiggleReflection1 AND 2


//Compute screen coordinates for the pixel on the screen from the position //of the point in -1/+1 projection space...

vec2 uv = screenUV (positionPS);//Normalize the 2 tangent vectors (interpolation made then not unit length)...

vec3 toLight = normalize (SHADER_INPUT toLightTS);vec3 toCamera = normalize (SHADER_INPUT toCameraTS);

//Compute perlin texture coord jiggle from 2 time varying texture coord functions...//Note1: When transforming texture coordinates, you see the inverse; //e.g., scale by 1/4 makes it 4 times bigger...//Note2: Time translate in opposite directions to give NET 0 velocity...vec2 direction1 = vec2 (1.0, 1.0); vec2 direction2 = vec2 (-1.0, -1.0); vec2 deltaUV1 = textureCoordinate * 11 + direction1 * wilfTime * 0.01;vec2 deltaUV2 = textureCoordinate * 16 + direction2 * wilfTime * 0.01; vec3 jiggle1 = sampleNormal (perlinSampler, deltaUV1) * 0.1;vec3 jiggle2 = sampleNormal (perlinSampler, deltaUV2) * 0.2; vec3 jiggle = jiggle1 + jiggle2;float zInRange01 = abs (positionPS.z / positionPS.w);jiggle *= lerp (0.25, 1.0, zInRange01); //close 0.25; far 1);

Finally, Shader for Final ResultFinally, Shader for Final Result


//Sample the reflection texture directly with a little less jiggle for refraction...vec3 incidentRayTS = -toCamera; vec3 normalTS = vec3 (0.0, 0.0, 1.0);vec4 reflectedPixel = texture2D (reflectionSampler, uv + jiggle.xy);vec4 refractedPixel = texture2D (refractionSampler, uv + jiggle.xy * 0.5);

//Colorize the refracted pixel...refractedPixel = colorize (refractedPixel);

//Determine how much of each pixel to use; more refraction when near and looking //straight at hit; more reflection otherwise.

float howMuchOfReflectedTextureToUse = waterFresnelRatio (incidentRayTS, normalTS);

const bool usingRefraction = true; //howMuchOfReflectedTextureToUse = 0;gl_FragColor = usingRefraction

? lerp (refractedPixel, reflectedPixel, howMuchOfReflectedTextureToUse)

: vec4 (reflectedPixel.rgb, howMuchOfReflectedTextureToUse);

Finally, Shader for Final ResultFinally, Shader for Final Result


vec4 colorize (vec4 pixel) {//Crytex coloring... From paper “Generic refraction simulation, //Tiago Sousa, Crytex, Chap 19, GPU Gems 2, 2005”.//Note: Using an alpha of 0 ensures the pixel’s alpha remains unchanged...//vec4 watercolor = vec4 (0.0, 0.15, 0.115, 0.0); //bluish//vec4 watercolor = vec4 (0.92, 0.7, 0.81, 0.0); //pinkishPurplevec4 watercolor = vec4 (0.0, 1.0, 1.0, 0.0); //cyan//vec4 watercolor = vec4 (1.0, 0.0, 1.0, 0.0); //purplereturn pixel + watercolor;

}

Colorizing The ResultColorizing The Result


95.450195.4501

Waterflow


• Flow can be encoded via vectors that indicate the direction and speed of the flow.

3D flow: rgb for x/y/z direction, a for speed

2D flow: rg for x/y direction, b for speed, a for something else (e.g., transparency)

FlowFlow


• Flow simulators use complex physics to create flows; e,g, ibfvs (source code NOT available).

Flow SimulatorsFlow Simulators

Image Based Flow Visualization, Jarke J. vanWijk, Sample code, Supplemental material, ACM SIGGRAPH 2002.

Demo


• A simple flow editor that can be used to create flow textures...

• Without one, we need to manually encode colors to represent flow...

Potential ProjectPotential Project

-(all right) NO right all right

-(all up) NO up all up


An Example Flow TextureAn Example Flow Texture

From frans van hoesel, university of groningen. 2010


Demo 1Demo 1

Kyle HaywardI'm a graphics programmer at Human Head Studios.

Kyle Hayward, graphics programmer, Human Head Studios.

Implemented contents of Valve’s SIGGRAPH 2010presentation of water flow in Left For Dead 2 and Portal 2

Demo

No source code (or shader)

http://www.blogger.com/profile/00654406875137720609


Demo 2Demo 2


Demo

Sort of source code (with shader)


Advantages of Newer TechniqueAdvantages of Newer Technique


Disadvantages of Valve technique:

• Only works for non-directional wave patterns. If you looked at a still from the animation, you couldn’t see in which direction the waves are travelling.

• Changing blendfactor introduces a visible pulsing behaviour. Vlachos (Valve) did hide that by introducing noise.


• A flow texture + a sampling texture (here, it’s perlin waves as a normal map).

Generally, 2 textures are involved...Generally, 2 textures are involved...

It’s as if the sample map were moved around in the flow direction...

flow texture sampling texture


• Flow is easiest to manifest in a game by translating texture coordinates in an infinite texture; using wrap rather than clamp mode...

• Imagine what happens to 2 neighbors as translations are performed in 2 slightly different directions...

The basic ProblemThe basic Problem

Over time, the pixel locations diverge.

Initially continuous,ultimately discontinuous

time 0

time 1

time 2

time 3


• Try to snap the texture coordinate back to the start if it goes too far (continual loop back)...

• Can be made to work if flow is all in same direction...

• Otherwise, the snapping needs to be hidden with suitable noise...

Early ApproachesEarly Approaches

Artifacts are always visible


• Scroll an entire block of pixels together IN ONE FLOW DIRECTION...

New IdeaNew Idea

time 0

time 1

time 2

time 3

no pixel divergence

in long term, pattern will repeat

left pixel flow right pixel flow


• By gridding the original flow texture at the corner points via virtual grid (no actual cutting up).

How it’s DoneHow it’s Done

This grid size is too large (10 times smaller than this)

pixels inside access the flows of the corner points

via an algorithm focused on blending


• Based on an offset tile pattern with non-uniform tile contribution per region.

• Based on a centered tile pattern with a uniform tile contribution per region.

Two Techniques For Combining Tile FlowTwo Techniques For Combining Tile Flow

frans van hoesel

lalonde (a modification of frans van hoesel)


The Problem: How to Interpolate between Tiles?The Problem: How to Interpolate between Tiles?

C C C C D D D DC C C C D D D DC C C C D D D DC C C C D D D DA A A A B B B BA A A A B B B BA A A A B B B BA A A A B B B B

C D

A B

Each flow vector has an entire tile associated with it...Assume 4x4 entries per tile.

If a tlle is 4x4, we want to find a 4x4 subtile of the 16x16

entries on which to enforce continuity.

Each flow quad processes the same subtile pattern; so the entire water flow is

processed SIMILARLY.


The Problem: how to Interpolate between Tiles?The Problem: how to Interpolate between Tiles?



offset tile, non-uniform contribution centered tile, uniform contribution



Strategy 1: Offset tile, Non-uniform contributionStrategy 1: Offset tile, Non-uniform contribution

Flow vector sample is

offset left and down from

center of A-tile

2 tiles contribute to these regions

All 4 tiles contribute to this region

Only tile A contributes

to this region


Strategy 2: Centered tile, Uniform contributionStrategy 2: Centered tile, Uniform contribution

C C C C D D D DC C C C D D D DC C C C D D D DC C C D D DA A A B B BA A A A B B B BA A A A B B B BA A A A B B B B

Linearly interpolate all 4 tiles

across the entire region

Flow vector sample from

center of tiles


The 3 textures UsedThe 3 textures Used

color texture sampling texture(perlin waves)

flow texture

It’s actually flowing horizontally (would be

better if vertical)

One quad is being drawn


• The demo is built with “open scene graph” which we had to download and integrated with the existing software...

• Uses a file called “hpcvwater.osg” (see next slide). One way of instrumenting the code is to use different shaders in the “.osg” file

“hpcvwater.osg”

“wilf-lalonde-flow.osg”

Instrumenting the DemoInstrumenting the Demo


The “open scene graph” file “hpcvwater.osg”...The “open scene graph” file “hpcvwater.osg”...

ProxyNode {StateSet {

DataVariance STATICProgram {

DataVariance STATICnum_shaders 2Shader {

DataVariance STATICtype VERTEXfile "shaders/hpcv-water-tile.vert“

}Shader {

DataVariance STATICtype FRAGMENTfile "shaders/hpcv-water-tile.frag“

}} Uniform {name "Normal0Map“ int 0}Uniform {name "cubeMap“ int 1}Uniform {name "colorMap“ int 3}Uniform {name "flowMap“}

}FileNameList {"geometry/hpcv-water-tile-test.osg“}num_children 1

}reformatted to fit

Contains information about cube map and sampling

texture “w1-2-grey.tga” for perlin waves

Use different shader here

wilf-lalonde-water-tile

wilf-lalonde-water-tile-testUses sampling texture

“wilf-vertical-perlin-wave.tga”for perlin waves(not important)

Use different osg file here


• Quick look at original shaders.

Vertex shader will be unchangedPixel shader is fairly complex

• Special details applicable to flows...The texture coordinates being usedHow virtual gridding is doneExperiment to see “per pixel flow”.Building a flow rotation matrix

• Discussion of “frans van hoesel” technique.• Discussion of “lalonde” technique.

Presentation OrderPresentation Order


95.450195.4501

A Quick Look at

Original Shaders


#version 120//Vertex shader

varying vec3 Normal;varying vec3 EyeDir;

varying vec2 texNormal0Coord;varying vec2 texColorCoord;varying vec2 texFlowCoord;

uniform float osg_FrameTime;uniform mat4 osg_ViewMatrixInverse;varying float myTime;void main () {

gl_Position = ftransform ();Normal = normalize (gl_NormalMatrix * gl_Normal);

vec4 pos = gl_ModelViewMatrix * gl_Vertex;EyeDir = vec3 (osg_ViewMatrixInverse * vec4 (pos.xyz, 0));texNormal0Coord = gl_MultiTexCoord1.xy;texColorCoord = gl_MultiTexCoord3.xy;texFlowCoord = gl_MultiTexCoord2.xy;myTime = 0.01 * osg_FrameTime;

}

The Vertex Shader Drawing a QuadThe Vertex Shader Drawing a Quad

0.0

1.0

1.0 All 3 sets ofcoordinates

go from 0 to 1

VERIFIED LATER

texNormal0Coord;texColorCoord;

vec2 texFlowCoord;

model view projection matrix

model view matrix

world space

1000 milliseconds = 1 second


• We won’t try to understand it in detail... Just point out some things... MORE DETAILS LATER.

The Pixel ShaderThe Pixel Shader

See code in actual shader


95.450195.4501

Some Issues

Resolved With

Experiments


Verifying The Texture Coordinates ClaimVerifying The Texture Coordinates Claim

//gl_FragColor = vec4 (texNormal0Coord, 0, 1); return; //0 to 1 in x, 0 to 1 in y.//gl_FragColor = vec4 (texColorCoord, 0, 1); return; //0 to 1 in x, 0 to 1 in y.gl_FragColor = vec4 (texFlowCoord, 0, 1); return; //0 to 1 in x, 0 to 1 in y.

First 3 lines draw the 2D color; red + green;

no blue

0 to 1 for red

0 to 1 for green


vec2 samplePerPixelFlowVector (vec2 uv) {vec4 pixel = texture2D (flowMap, uv).rgba;return pixel.b * (pixel.rg - 0.5); //rg for x/y direction, b for speed, a for

transparency. }#define sampleFlowAlpha(uv) (texture2D (flowMap, uv).a)

#define time (myTime * 2.0) //It's deciseconds...

vec2 sampleTimeAffectedNormal (vec2 uv, vec2 flowVector) {

//Build the rotation matrix that scales and rotates the tile we're in...mat2 rotationMatrix = mat2 (flowVector.x, -flowVector.y, flowVector.y,

flowVector.x);float timeBasedCoordinate = rotationMatrix * uv - vec2 (time, 0.0);return texture2D (normalMap, timeBasedCoordinate).rg;

}m

void main () {float flowTileFactor = 35.0;vec2 scaledFlowCoordinate = texFlowCoord * flowTileFactor;vec2 flowVector = samplePerPixelFlowVector (texFlowCoord);vec2 normal = sampleTimeAffectedNormal (scaledFlowCoordinate, flowVector);float alpha = sampleFlowAlpha (texFlowCoord); gl_FragColor = vec4 (normal * alpha, 0.0, 1.0);

}

Per Pixel Flow ExperimentPer Pixel Flow Experiment

Above code has since been redone

How ROTATION is handled is described LATER

Normal rotated into flow direction


Not Particularly ContinuousNot Particularly Continuous


• Still drawing ONE QUAD (no actual cutting up).

How Virtual Gridding Is AchievedHow Virtual Gridding Is Achieved

This grid size is too large (10 times smaller than this)


• A 0/1 texture coordinate t can be snapped to one of “n” virtual grid points via floor (t*n) / n.

• Inside the grid, t can be turned into a 0/1 grid coordinate via fract (t * n).

Getting a Virtual GridGetting a Virtual Grid

floor and fract are shader functions...

A more detailed explanation coming up


Example gridding by 3Example gridding by 3

t 0 => 1

range

t * n 0 => n

floor (t * n) 0 => n

0.75

2.25

2.0

floor (t * n) / n

0 => 1

0.66

Let n = 31.0

3.0

3.0

1.0

0.0

0.0

fract (t * n)

0.25

0 => 1 0 => 1

0 => 1 grid point

coord within grid


• Access the flow map to get the flow direction ONLY AT THE VIRTUAL CORNER POINTS... (4 flow vectors)

How We Use The Virtual GridHow We Use The Virtual Grid

C D

A B

sample point

• For each corner, use the sample point and the CONSTANT flow and time to get a value from the texture being sampled.


vec2 tile (vec2 uv, float tiles) {return floor (uv) / tiles;}

vec2 sampleFlowVector (vec2 uv) { //return the decoded vector}vec2 sampleFlowVector (vec2 uv, float tiles, vec2 tileOffset) {//at tile vertex... return sampleFlowVector (tile (uv + tileOffset, tiles));}

#define flowTileFactor 35.0 vec2 scaledFlowCoordinate = texFlowCoord * flowTileFactor;

vec2 flowA = sampleFlowVector (scaledFlowCoordinate, flowTileFactor, vec2 (0.0, 0.0));vec2 flowB = sampleFlowVector (scaledFlowCoordinate, flowTileFactor, vec2 (1.0, 0.0));vec2 flowC = sampleFlowVector (scaledFlowCoordinate, flowTileFactor, vec2 (0.0, 1.0));vec2 flowD = sampleFlowVector (scaledFlowCoordinate, flowTileFactor, vec2 (1.0, 1.0));

Samples Flow Only At Virtual Grid PointsSamples Flow Only At Virtual Grid Points

e.g., 0.7 * 35 = 24.5

e.g., 24.5 + 1 = 25.5

e.g., 24/35 or 25/35


How Rotation Is HandledHow Rotation Is Handled

flow vector = [a, b]

up

right

a

b

-a

b

[0, 1] * ? ? = [a, b]

? ?

[1, 0] * ? ? = [b, -a]

? ?

90 to flow vector = [b, -a] (DEDUCED FROM )

up

right


How Rotation Is HandledHow Rotation Is Handled

[0, 1] * ? ? = [a, b]

a b

[1, 0] * b -a = [b, -a]

? ?

up

right

• But to transform texture coordinates, you need the inverse of what you want; i.e. transpose...

b -a

a b

b a

-a b


95.450195.4501

Discussion of

“frans van hoesel”

Approach


• Resets A,B,C,D to different values depending on which quarter the sample point is in.

How The Pixel Shader Works...How The Pixel Shader Works...

AAAA

ABAB

CCAA

CDAB

A AB x < .5 ? A : BC y < .5 ? A : C.D

A (if x < .5 & y < .5), B (if x >= 0.5 & y < .5) C (if x < 0.5 & y >= .5) D (otherwise).

A B

C D

This is the pattern we described earlier


• Brace blend (straight lines less smooth) goes to 0 at ends...

Make Use of 2 Blend FormulasMake Use of 2 Blend Formulas

0.0 +1.0

+1.0

0.0

• Crossover blend

Use more of the right side as you approach the left side (use more of the left side as you approach the right side).


// ff is the factor that blends the tiles.vec2 ff = abs(2*(fract(t*n))-1) - 0.5; // take a third power, to make the area with more or less equal //contribution of more tile biggerw = 0.5-4*ff*ff*ff;

Analysis of the Code Producing the Brace BlendAnalysis of the Code Producing the Brace Blend

range

fract (t*n) 0 => 1

2*(fract(t*n)) - 1 -1 => 0 => +1

abs (2*(fract(t*n)) - 1) 1=> 0=>1

ff = abs (2*(fract(t*n)) - 1) - 0.5 0.5=>-0.5=>0.5

0.53 = 0.1254* 0.125 = 5.0

4ff3 0.5=>-0.5=>0.5

linear

w = 0.5 - 4ff3 0.0=>1.0=>0.0

nonlinear

-0.5 +0.5

+1.0

Looks like a curly bracebut a little rounder(see blowup slide)


vec2 AB = w.x * A + (1-w.x) * B;vec2 CD = w.x * C + (1-w.x) * D;

vec2 Result = w.y * AB + (1-w.y) * CD;

Analysis of Code Producing the Crossover BlendAnalysis of Code Producing the Crossover Blend

NOTE CROSSOVER

When w.x == 0, have B or D resp.

When w.y == 0, should have CD

AAAA

ABAB

CCAA

CDAB

A B

C D


• Output of brace blend w.x controls crossover

How the Blend Formulas Are CombinedHow the Blend Formulas Are Combined

0.0 +1.0

+1.0

0.0

Use right Use left Use right


The Brace + Crossover BlendThe Brace + Crossover Blend

AAAA

ABAB

CCAA

CDAB

A B

C D

0.0 +1.0 +1.0

0.0

use right use left use right

=>AAAA

ABAB

CCAA

CDAB

A B

C D

flipsbut no noticeable effect

Use left on leftand right on right (no effect again)

Final effect is to smoothlyinterpolate from C to C, then C to D.


The Special Pattern Described Made VisibleThe Special Pattern Described Made Visible

zoom in here

The author shows the A contribution with B, C, D set

to 0 so we can see the contribution of the single tile (this is not discernable under

normal blending)

AAAA

ABAB

CCAA

CDAB


More Noticeable in Motion (zoomed in portion)More Noticeable in Motion (zoomed in portion)

AAAA

ABAB

CCAA

CDAB


95.450195.4501

Reimplementing a

SIMPLER version

The “lalonde” Approach


Encapsulating frans van hoesel’s water shadingEncapsulating frans van hoesel’s water shading

vec4 worldColor (float alpha, vec2 normal2D, float normalMapScale) {//A straightforward excapsulation of the code found in frans van hoesel's

shader...//To make the water more transparent, scale the normal with the transparencynormal2D *= 0.3 * alpha * alpha;//Assume the normal of plane is 0,0,1 and produce the normalized sum of adding normal2D to it.

vec3 normal3D = vec3 (normal2D, sqrt (1.0 - dot (normal2D, normal2D)));vec3 reflectDirection = reflect (EyeDir, normal3D);vec3 envColor = vec3 (textureCube (cubeMap, -reflectDirection));

//Very ugly version of fresnel effect but it gives a nice transparent water, but not too transparent.

float myangle = dot (normal3D, normalize (EyeDir));myangle = 0.95 - 0.6 * myangle * myangle;

//Blend in the color of the plane below the water//Add in a little distortion of the colormap for the effect of a refracted view of

the//image below the surface (this isn't really tested, just a last minute addition//and perhaps should be coded differently

//The correct way, would be to use the refract routine, use the alpha channel for depth of

// the water (and make the water disappear when depth = 0), add some watercolor to the colormap

// depending on the depth, and use the calculated refractdir and the depth to find the right

// pixel in the colormap.... who knows, something for the next versionvec3 base = texture2D (colorMap, texColorCoord +

vec2 (normal3D * (0.03*alpha/normalMapScale))).rgb;return vec4 (lerp (base,envColor, myangle*alpha), 1.0);}


#define lerp mix

vec2 interpolateQuad (vec2 A, vec2 B, vec2 C, vec2 D, vec2 position) {//Position.x ranges from 0 to 1 from A to B and C to D and //position.y ranges from 0 to 1 from A to C and B to D.

vec2 AB = lerp (A, B, position.x);vec2 CD = lerp (C, D, position.x);

vec2 ABCD = lerp (AB, CD, position.y);return ABCD;

}

A Simple QUAD InterpolatorA Simple QUAD Interpolator

A

C

B

D

AB

CD

ABCD position.y

position.x

Just QUAD interpolate the 4 values

Will be used to interpolate 4 normals


vec2 sampleDynamicNormal (vec2 scaledFlowCoordinate, vec2 flowVector) {

//Build the rotation matrix that scales and rotates the tile we're in //and scroll the texture coordinate...

float scale = 1.0 - (2.0 * length (flowVector)); //flowVector = normalize (flowVector);

mat2 rotationMatrix = mat2 (flowVector.y, flowVector.x, -flowVector.x, flowVector.y);

return texture2D (normalMap, rotationMatrix * scaledFlowCoordinate - vec2 (0.0, time * scale * 2.0)).rg * sqrt (scale) * 2.0;

}

Sampling a Normal At Time Varying LocationsSampling a Normal At Time Varying Locations

Algorithm: 1. Rotate the texture to align with flow vector (an inverse)2. Translate forward by time in the y direction (an inverse)3. Apply scale fudge factors to both time and the answer.

Scale by 1 when NO FLOW Scale by MIN when MAX FLOW (an inverse)

SCALE is an effort to give more speed to the flow

vector.


void main () {vec2 scaledFlowCoordinate = texFlowCoord * flowTileFactor;vec2 flowA = sampleFlowVector (scaledFlowCoordinate, flowTileFactor, vec2 (0.0,

0.0));vec2 flowB = sampleFlowVector (scaledFlowCoordinate, flowTileFactor, vec2 (1.0,

0.0));vec2 flowC = sampleFlowVector (scaledFlowCoordinate, flowTileFactor, vec2 (0.0,

1.0));vec2 flowD = sampleFlowVector (scaledFlowCoordinate, flowTileFactor, vec2 (1.0,

1.0));

vec2 scaledNormalFlowCoordinate = texFlowCoord * normalFlowTileFactor;vec2 normalA = sampleDynamicNormal (scaledNormalFlowCoordinate, flowA); vec2 normalB = sampleDynamicNormal (scaledNormalFlowCoordinate, flowB); vec2 normalC = sampleDynamicNormal (scaledNormalFlowCoordinate, flowC); vec2 normalD = sampleDynamicNormal (scaledNormalFlowCoordinate, flowD);

float alpha = sampleFlowAlpha (texFlowCoord);vec2 normal2D = interpolateQuad (normalA, normalB, normalC, normalD,

fract (scaledFlowCoordinate));gl_FragColor = worldColor (alpha, normal2D, normalFlowTileFactor);

}

The ShaderThe Shader


• Cubes maps and their use in environment mapping + skyboxes (mostly obsolete).

• Reflection + refraction + fresnel term for interpolating between the two.

• Drawing world multiple times and blending the results for water that has both.

• Flow techniques...

ReviewReview

wilf lalonde ©2012 comp 4501 95.4501 water. wilf lalonde ©2012 comp 4501 four properties that...

Documents

sampling cube map wilf

dynamic cube maps

static cube maps

rinwilf lalonde

float3 normal

techniqueswilf lalonde

float scale

float power