cs 480/680: game engine programming animationsanti/teaching/2013/cs...cs 480/680: game engine...

CS 480/680: GAME ENGINE PROGRAMMING ANIMATION

2/21/2013 Santiago Ontañón santi@cs.drexel.edu https://www.cs.drexel.edu/~santi/teaching/2013/CS480-680/intro.html

Game AI in Educational Game A CS RA @ 20 hrs/week for two-terms in the Digital Media program. Experience Management for educational games Skills:

* basic knowledge in AI * C# and/or JavaScript * Unity 3D experience is a plus

If you are interested, contact me.

Outline • Student Presentations • Cel Animation • Skeletal Animation • Skinning • Project Discussion

Outline • Student Presentations • Cel Animation • Skeletal Animation • Skinning • Project Discussion

Student Presentations •  Steven Bauer:

•  “Real-time Shadows on Complex Objects”

•  Christopher Miller: •  “Real-Time Strategy Network Protocol”

•  Joseph Muoio: •  “Search-based Procedural Content Generation”

•  James Rodgers •  “Terrain Analysis in an RTS – The Hidden Giant”

•  Justin Weaver: •  “Coping with Friction in Dynamic Simulations”

Outline • Student Presentations • Cel Animation • Skeletal Animation • Skinning • Project Discussion

Game Engine Architecture

HARDWARE

DRIVERS

OS

SDKs

Platform Independence Layer

Utility Layer

Resource Management

Game Engine Functionalities

Game Specific

Game Engine

Dependencies

Game Engine Architecture

Rendering Engine

Animation Engine

Collisions

Physics Audio Subsystem

Online Multiplayer

Profiling & Debugging

Gameplay Foundations (Game Loop)

Artificial Intelligence Scripting

Game Engine Architecture

Rendering Engine

Animation Engine

Collisions

Physics Audio Subsystem

Online Multiplayer

Profiling & Debugging

Gameplay Foundations (Game Loop)

Artificial Intelligence Scripting

Animation • Mostly for characters, but used for any animated object in

the game • Animation is DIFFERENT from “movement”:

•  Movement refers to game objects changing location and orientation in the game world

•  Animation refers to game objects changing shape, or appearance while potentially not changing location nor orientation.

Animation Types • Cel Animation (applicable to bitmap graphics)

• Skeletal Animation (applicable to all kind of graphics)

• Per-Vertex Animation (applicable to vector-based representations)

• Skinned Animation (applicable to vector-based representations)

Cel Animation Examples • Classic pixel-art 2d animations:

• Graphic artist draws different frames (“cels”) for each animation, which the game engine displays in sequence

•  http://www.youtube.com/watch?v=ePJsX2YdqAs

Cell-Animation Engine • Basic concepts:

•  Sprite: (recall from the “rendering lecture”): •  Bitmap •  Hotspot

•  Clip: •  A scripted sequence of sprites that represents a specific animation •  For example: “walking right”, or “crouching”

•  The animation engine allows the game engine to assign a “clip” to a game object. The clip determines the sprite to be drawn at each game cycle.

Cell-Animation Engine • Example code

Class Sprite { Bitmap bm; int hot_x; int hot_y; Void draw(int x, int y);

}

Class Clip { List<Sprite> sprites; List<Integer> times; int local_time; Boolean looping; Sprite update();

}

These lists define the animation sequence

A clip keeps an internal clock, that is incremented

each time the “update” function is called.

Update advances the clock, checks which is the

sprite that should be displayed at the current

time, and returns it.

Cell-Animation Engine • Example code

Class Sprite { Bitmap bm; int hot_x; int hot_y; Void draw(int x, int y);

}

Class Clip { List<Sprite> sprites; List<Integer> times; int local_time; Boolean looping; Sprite update();

}

times = [ 0, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, 30 ]

bitmaps= [ ]

Cell-Animation Engine • Example code

Class Sprite { Bitmap bm; int hot_x; int hot_y; Void draw(int x, int y);

}

Class Clip { List<Sprite> sprites; List<Integer> times; int local_time; Boolean looping; Sprite update();

}

times = [ 0, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, 30 ]

bitmaps= [ ]

This is oversimplistic. More expressive ways to define

clips (allowing randomness, etc.) can be

defined.

Cell-Animation Engine • As part of your “AI” or “Scripting” code, each game object

will have an “update” function.

• Such update function is responsible for assigning “clips” to each game object

•  The rendering module calls the clips to determine which sprite to draw.

Outline • Student Presentations • Cel Animation • Skeletal Animation • Skinning • Project Discussion

Skeletal Animation • Characters made up of rigid parts combined together

using a hierarchical “skeleton” •  This can be done in either 2D or 3D:

•  2D: http://www.youtube.com/watch?v=5_wzNRwyGD0 •  3D: http://www.youtube.com/watch?v=Vb3L7Odrcdo

Skeleton •  Hierarchy of “rigid pieces” known as bones. •  Example for a human character:

Pelvis LowerSpine

UpperSpine RightArm

RightForearm RightHand

LeftArm LeftForearm

LeftHand RightThigh

RightLeg RightFoot

LeftThigh LeftLeg

LeftFoot

Skeleton •  In code, a skeleton can be represented as:

Class Skeleton { List<Bone> bones;

}

Class Bone{ Pose defaultPose;

String name; int parent; List<int> children;

}

Id: 0, Name: Pelvis, parent: -1, children: [1,9,12] Id: 1, Name: LowerSpine, parent: 0, children: [2] Id: 2, Name: UpperSpine, parent: 1, children: [3,6] Id: 3, Name: RightArm, parent: 2, children: [4] Id: 4, Name: RightForearm, parent: 3, children: [5] Id: 5, Name: RightHand, parent: 4, children: [] Id: 6, Name: LeftArm, parent: 2, children: [7] Id: 7, Name: LeftForearm, parent: 6, children: [8] Id: 8, Name: LeftHand, parent: 7, children: [] Id: 9, Name: RightThigh, parent: 1, children: [10] Id: 10, Name: RightLeg, parent: 9, children: [11] Id: 11, Name: RightFoot, parent: 10, children: [] Id: 12, Name: LeftThigh, parent: 1, children: [13] Id: 13, Name: LeftLeg, parent: 12, children: [14] Id: 14, Name: LeftFoot, parent: 13, children: []

Skeleton •  In code, a skeleton can be represented as:

Class Skeleton { List<Bone> bones;

}

Class Bone{ Pose defaultPose;

String name; int parent; List<int> children;

}

Id: 0, Name: Pelvis, parent: -1, children: [1,9,12] Id: 1, Name: LowerSpine, parent: 0, children: [2] Id: 2, Name: UpperSpine, parent: 1, children: [3,6] Id: 3, Name: RightArm, parent: 2, children: [4] Id: 4, Name: RightForearm, parent: 3, children: [5] Id: 5, Name: RightHand, parent: 4, children: [] Id: 6, Name: LeftArm, parent: 2, children: [7] Id: 7, Name: LeftForearm, parent: 6, children: [8] Id: 8, Name: LeftHand, parent: 7, children: [] Id: 9, Name: RightThigh, parent: 1, children: [10] Id: 10, Name: RightLeg, parent: 9, children: [11] Id: 11, Name: RightFoot, parent: 10, children: [] Id: 12, Name: LeftThigh, parent: 1, children: [13] Id: 13, Name: LeftLeg, parent: 12, children: [14] Id: 14, Name: LeftFoot, parent: 13, children: []

“defaultPose” sometimes called the “bindPose”

Skeleton Poses •  The pose of a bone is defined by (relative to the parent

bone): •  Location •  Orientation •  Scale

•  Typically represented as a SQT structure: Scale (1 number, or a vector), rotation (1 Quaternion), Translation (1 vector) •  The SQT structure is preferred to the typical 4x4 matrix used for

rendering, since it’s better for “blending” purposes (more on this later)

Skeleton Poses •  The pose of a bone is defined by (relative to the parent

bone): •  Location •  Orientation •  Scale

•  Typically represented as a SQT structure: Scale (1 number, or a vector), rotation (1 Quaternion), Translation (1 vector) •  The SQT structure is preferred to the typical 4x4 matrix used for

rendering, since it’s better for “blending” purposes (more on this later)

Class Pose { Quaternion r; Vector t; double s;

}

Key Poses • Remember: in “cel animation” the animation was defined

by different bitmaps, annotated with the times at which we need to change from one bitmap to another

•  In Skeletal animation, the idea is the same: •  We define “key poses” (analogous to the bitmaps in cel animation) •  We define the times at which the character should be in each key

pose

• A key pose: list of poses for each bone (translation, rotation, scale)

Pose Interpolation • Clip: sequence of key poses at times t1, t2, …, tn

•  For each time t such that ti < t < tj •  The animation engine can interpolate between the poses defined at

ti and tj •  Interpolation generates intermediate poses

• Since we can interpolate poses, we can play animations at any speed, or even in reverse: •  Game Time: time as maintained by the game loop •  Animation Time: time as maintained internally by the clip

Pose Interpolation •  Notice that each pose is basically 10 numbers:

•  3 for translation •  4 for rotation •  3 for scaling

•  A clip can be seen as the definition of 10 functions over time:

-2

-1

0

1

2

3

4

5

0 2 4 6 8 10 12 14 16 18

Tx

Ty

Tz

Qx

Qy

Qz

Qw

Sx

Sy

Sz

Pose Interpolation •  Notice that each pose is basically 10 numbers:

•  3 for translation •  4 for rotation •  3 for scaling

•  A clip can be seen as the definition of 10 functions over time:

-2

-1

0

1

2

3

4

5

0 2 4 6 8 10 12 14 16 18

Tx

Ty

Tz

Qx

Qy

Qz

Qw

Sx

Sy

Sz

Interpolation typically just means linearly interpolating

between the values specified in the key poses.

Clip • Example clip classes:

Class JointPose { List<Pose> poses;

}

Class Clip{

List<JointPose> keyPoses; List<Double> times; double globalTimeAtStart; double localTime; double timeSpeed; JointPose update(double t);

}

A Clip maintains an internal clock and animation speed. In

this way, it can properly perform interpolation.

Clip • Example clip classes:

Class JointPose { List<Pose> poses; Event event;

}

Class Clip{

List<JointPose> keyPoses; List<Double> times; double globalTimeAtStart; double localTime; double timeSpeed; JointPose update(double t);

}

Additionally, we might want to annotated key poses with events.

Like “footstep”, to trigger any game event we might need (such

as playing some sound).

Or we could even add color changes, etc.

Outline • Student Presentations • Cel Animation • Skeletal Animation • Skinning • Project Discussion

Per-Vertex Animation • Modifying the vertexes and triangles of a mesh

(deforming) to create realistic animations •  Typically used for facial animations:

•  http://www.youtube.com/watch?v=IgRia5mRsWU (3:30)

Skinned Animation • Hybrid between per-vertex animation and skeletal

animation. Consists on having a skeleton plus a deformable skin around it (skeleton is not rendered)

• Used typically for skin and cloth: •  Oblivion: http://www.youtube.com/watch?v=n3Lje6CQzKA •  Spore: http://www.youtube.com/watch?v=FQIF2_gxgUg

Skinning •  The skin is just a triangle mesh assigned to a skeleton • Each skin mesh has:

•  A list of bones to which the skin is bound •  For each vertex in the mesh:

•  A weight for each of the bones: how much influence each bone has on this vertex

Bones Skin

Skinning • Vertex coordinates of the skin defined in coordinates

relative to the bone

y

x

Skinning • We first obtain the transformation matrix of the bone, and

then use it on each of the vertexes of the skin for rendering.

y

x

~p

Mb

Skinning • We first obtain the transformation matrix of the bone, and

then use it on each of the vertexes of the skin for rendering.

y

x

~p

Mb

~pw = Mb~p

Skinning •  In general, vertexes in the skin can be bound to more than

one bone

y

x

~p1

x

y

~p2

Skinning •  In general, vertexes in the skin can be bound to more than

one bone

y

x

~p1

x

y

~p2

~pw = w1M1~p1 + w2M2~p2

Skinning •  In general, vertexes in the skin can be bound to more than

one bone

y

x

~p1

x

y

~p2

~pw = w1M1~p1 + w2M2~p2

In order to prevent having to store the coordinates of each vertex

relative to each bone, the game engine just stores the vertexes once, and also a transformation

matrix per skin piece and bone, to map skin coordinates to bone

coordinates.

Skinning •  In general, vertexes in the skin can be bound to more than

one bone

[0, 1] [0.5, 0.5]

[1, 0]

Binding weights, vary gradually over the skin, to obtain the

desired “skin” effect:

[0.8, 0.2] [0.9, 0.1]

Skinning •  In general, vertexes in the skin can be bound to more than

one bone

[0, 1] [0.5, 0.5]

[0.8, 0.2] [0.9, 0.1] [1, 0]

Skinning •  In general, vertexes in the skin can be bound to more than

one bone

[0, 1] [0.5, 0.5]

[0.8, 0.2]

[0.9, 0.1]

[1, 0]

Skinning •  In general, vertexes in the skin can be bound to more than

one bone •  This is how it would look like with another skin piece • Notice the smooth transition at the elbow

Skinning •  In general, vertexes in the skin can be bound to more than

one bone •  This is how it would look without skinning • Notice the abrupt transition at the elbow

Skinning •  It can be used both in 3D and 2D •  It can be used for body parts, but also for facial

expressions (face bones)

Animation Blending • Given two animations, blend them into one • Uses:

•  Smooth transitions between animations (blend the end of one with the beginning of the other)

•  Gradual effects: if we have a walking animation, and an “injured walking” animation, we can produce different blends, for intermediate degrees of injury.

Animation Blending • Most common method: Linear interpolation (LERP)

• Compute the poses for both animations at the current time frame, and interpolate between them (like we did for pose animation).

• We just need a weight w, which initially is set to 1, and gradually goes down to 0, to transition from one animation to the other.

The Animation Engine

Rendering Engine

Animation Engine

Collisions

Physics Audio Subsystem

Online Multiplayer

Profiling & Debugging

Gameplay Foundations (Game Loop)

Artificial Intelligence Scripting

Game State

The Animation Engine

Rendering Engine

Animation Engine

Collisions

Physics Audio Subsystem

Online Multiplayer

Profiling & Debugging

Gameplay Foundations (Game Loop)

Artificial Intelligence Scripting

Scripting or AI modules set the current animation clip (of clip

blends) to be played to a particular game object.

Game State

Game State

The Animation Engine

Rendering Engine

Animation Engine

Collisions

Physics Audio Subsystem

Online Multiplayer

Profiling & Debugging

Gameplay Foundations (Game Loop)

Artificial Intelligence Scripting

Animation engine plays the clips, determines the poses, and skins the objects. The output is a set of transformation matrices that the rendering engine can take for

rendering each object/skin

Game State

The Animation Engine

Rendering Engine

Animation Engine The 3 main steps for in

the animation pipeline: 1)  Run clips

2)  Determine poses 3)  Skinning

The output is the

transformation matrices for each bone (and a set of matrices plus weights

for each skin vertex)

Run the clips, and get the key poses

Key pose blending, obtain the global pose

Skinning, obtain the correct transformations for

each vertex

Links to Interesting Game Videos

• Gish:

•  http://www.youtube.com/watch?v=5WzGQQOIcp8

• Locoroco: •  http://www.youtube.com/watch?v=QfGrX0zzVp0

• Aquaria: •  http://www.youtube.com/watch?v=YqY9mDOw-UI

• Non-rigid physics demo: •  http://www.youtube.com/watch?v=KppTmsNFneg

Outline • Student Presentations • Cel Animation • Skeletal Animation • Skinning • Project Discussion

Remember that next week: •  Third Project Deliverable:

•  First Demo of your Game Engine: •  If you are a in-class student, you’ll show it to the class. If you are an on-line group,

you’ll send me a short video and I’ll play it in class. •  Main functionalities should be there (it’s fine if it has bugs, crashes, etc., but I need to

see that most of the code is there) •  It needs to run and be playable (even if gameplay is very simple) •  Do not worry if the GUI is terrible, or if there is no GUI. I care about the engine, not the

GUI J •  Source Code •  Document:

•  Updated Game Engine diagram (shortened version of Deliverables 1 and 2) •  Explain how each team member is addressing their part (one section per team

members.

•  Submission procedure: •  Email to (copy both):

•  Santiago Ontañón santi@cs.drexel.edu •  Stephen Lombardi sal64@drexel.edu

•  Subject: CS480-680 Project Deliverable 3 Group #