java.sun.com/javaone/sf | 2004 javaone sm conference | session ts-2121 1 advanced game development...
TRANSCRIPT
java.sun.com/javaone/sf
| 2004 JavaOneSM Conference | Session TS-21211
Advanced Game Development with the Mobile 3D Graphics API
Tomi Aarnio, Kari PulliNokia Research Center
| 2004 JavaOneSM Conference | Session TS-2121 2
Gain an insight to the Mobile 3D Graphics API and take your games to the next level
| 2004 JavaOneSM Conference | Session TS-2121 3
Prerequisites
• Fundamentals of 3D graphics
• OpenGL® or some other modern 3D API
• Java™ technology, preferably MIDP
| 2004 JavaOneSM Conference | Session TS-2121 4
Objectives
• Get a quick overview of the whole API
• Gain a deeper understanding of selected topics
• Learn practical tricks not found in the spec
| 2004 JavaOneSM Conference | Session TS-2121 5
Agenda
Why should I care?
Getting started
Low level features
Bitmaps and texturing
Scene graph
Dynamic meshes
Animation
| 2004 JavaOneSM Conference | Session TS-2121 6
Agenda
Why should I care?
Getting started
Low level features
Bitmaps and texturing
Scene graph
Dynamic meshes
Animation
| 2004 JavaOneSM Conference | Session TS-2121 7
Mobile 3D Graphics API (M3G)
• Also known as JSR-184
• Designed for mobile devices─ Primarily CLDC / MIDP─ But also CDC
Mobile 3D Graphics API
Java Virtual Machine (CLDC 1.1)
MIDP
Midlets
Graphics Hardware
| 2004 JavaOneSM Conference | Session TS-2121 8
Overcome the performance barrier
00.10.20.30.40.50.60.70.80.9
1
Vertex transformation Image downsampling
Rel
ativ
e sp
eed
Native code
CLDC 1.1 HI
Jazelle™
KVM
Benchmarked on an ARM9 processor
Native (C/C++) vs. Java on mobiles
| 2004 JavaOneSM Conference | Session TS-2121 9
Leverage hardware acceleration
• Benefits for developers─ Mask hardware differences (3D HW, FPU, DSP,…)─ Exploit any new hardware automatically
• Benefits for hardware vendors─ Gives a concrete target which features to offer
| 2004 JavaOneSM Conference | Session TS-2121 10
Speed up development cycles
• Separate content from code─ Quick prototyping without programming─ Artists create looks and behavior─ Export all in a binary file format─ Load the file, populate scene graph
• Commonly needed functionality built in─ High-level functionality such as scene graph,
keyframe animation
| 2004 JavaOneSM Conference | Session TS-2121 11
Why a new standard?
• OpenGL (ES) is too low-level─ Lots of Java code needed for simple things
• Java 3D™ API is too bloated─ A hundred times larger than M3G─ Does not fit together with MIDP─ Tried and failed, but…
• Now knew what we wanted!─ Basic Java 3D™ ideas: nodes, scene graph─ Add file format, keyframe animation─ Remain compatible with OpenGL ES
| 2004 JavaOneSM Conference | Session TS-2121 12
Agenda
Why should I care?
Getting started
Low level features
Bitmaps and texturing
Scene graph
Dynamic meshes
Animation
| 2004 JavaOneSM Conference | Session TS-2121 13
Key classes
World
Graphics3D
Loader
3D graphics context Performs all rendering
Scene graph root node
Can load individual objects and entire scene graphs (M3G and PNG files)
| 2004 JavaOneSM Conference | Session TS-2121 14
Graphics3D
• Contains global state─ Target surface, viewport, depth buffer─ Camera, light sources─ Rendering quality hints
• Each renderable object has its own local state─ Geometry and appearance (material, textures, etc.)─ Transformation relative to parent or world
| 2004 JavaOneSM Conference | Session TS-2121 15
Graphics3D: Rendering modes
• Retained mode─ Render a scene graph, rooted by the World─ Take the Camera and Lights from the World
• Immediate mode─ Render a branch or an individual node at a time─ Explicitly give the Camera and Lights to Graphics3D
| 2004 JavaOneSM Conference | Session TS-2121 16
Graphics3D: How-To-Use
• Bind a target to it, render, release the target
• Tip: Never mess with a bound target
• Tip: Graphics3D is a singleton (threads!)
void paint(Graphics g) {
myGraphics3D.bindTarget(g);
myGraphics3D.render(world);
myGraphics3D.releaseTarget();
}
| 2004 JavaOneSM Conference | Session TS-2121 17
Graphics3D: Rendering targets
Graphics Canvas
Image
CustomItem
Graphics3D Image2D
World
Can render to textures also
M3G
MIDP
| 2004 JavaOneSM Conference | Session TS-2121 18
• Everything is synchronous─ A method returns only when it’s done─ No separate thread for renderer or loader
• There are no callbacks─ No abstract methods, interfaces, or events─ Tip: Do not even try to override any methods
• Scene update is decoupled from rendering─ render : draws the scene, no side-effects─ animate : updates the scene to the given time─ align : applies auto-alignments, e.g., billboards
Things to keep in mind
| 2004 JavaOneSM Conference | Session TS-2121 19
“Hello, World”
import javax.microedition.midlet.MIDlet;import javax.microedition.lcdui.Display;import javax.microedition.lcdui.game.GameCanvas;import javax.microedition.m3g.*;
public class Player extends MIDlet{ public void pauseApp() {}
public void destroyApp(boolean b) {}
public void startApp() { PlayerCanvas canvas = new PlayerCanvas(true); Display.getDisplay(this).setCurrent(canvas); try { canvas.run(); } catch (Exception e) {} notifyDestroyed(); }}
A simplified animation player
| 2004 JavaOneSM Conference | Session TS-2121 20
“Hello, World”
class PlayerCanvas extends GameCanvas { PlayerCanvas(boolean suppress){super(suppress);}
public void run() throws Exception { Graphics3D g3d = Graphics3D.getInstance(); World w = (World) Loader.load("/file.m3g")[0]; long start, elapsed, time = 0; while (getKeyStates() == 0) { start = System.currentTimeMillis(); g3d.bindTarget(getGraphics()); try { w.animate(time); g3d.render(w); } finally { g3d.releaseTarget(); } flushGraphics(); elapsed = System.currentTimeMillis()-start; time += (elapsed < 100) ? 100 : (int)elapsed; if (elapsed < 100) Thread.sleep(100-elapsed); } }}
| 2004 JavaOneSM Conference | Session TS-212121
Demo
Animation playback
21
| 2004 JavaOneSM Conference | Session TS-2121 22
Agenda
Why should I care?
Getting started
Low level features
Bitmaps and texturing
Scene graph
Dynamic meshes
Animation
| 2004 JavaOneSM Conference | Session TS-2121 23
Renderable Objects
Mesh
Sprite3D
Made of triangle stripsBase class for meshes (two subclasses)
2D image placed in 3D spaceGood for labels, etc.
| 2004 JavaOneSM Conference | Session TS-2121 24
Mesh
Mesh VertexBuffer coordinates
normals
colors
texcoords
IndexBuffer
Appearance
Composed of submeshes
• Common buffer of vertex data
• Submesh has triangle strip indices to vertices
• One Appearance for each submesh
| 2004 JavaOneSM Conference | Session TS-2121 25
Appearance components
CompositingMode
Material colors for lightingCan track per-vertex colors
PolygonMode
Fog
Texture2D
MaterialBlending, depth bufferingAlpha testing, masking
Winding, culling, shadingPerspective correction hint
Fades colors based on distanceLinear and exponential mode
Texture matrix, blending, filteringMultitexturing: One Texture2D for each unit
| 2004 JavaOneSM Conference | Session TS-2121 26
Sprite3D
• Scaled mode for billboards, trees, etc.
• Unscaled mode for text labels, icons, etc.
Image2D
2D image with a position in 3D space
Sprite3D Appearance
Image2D
CompositingMode
Fog
| 2004 JavaOneSM Conference | Session TS-2121 27
Rendering tricks
• Use layers to impose rendering order─ Appearance contains a layer index (integer)─ Defines a global ordering for submeshes and sprites─ Tip: Disable z-buffering for sky boxes, use layers
• Tip: Maximize triangle strip length─ Even if it requires adding degenerate triangles─ Better benefits from vertex caching
| 2004 JavaOneSM Conference | Session TS-2121 28
Example: Rotating cube
// Corners of a cube as (X,Y,Z) tripletsstatic short[] cubeVertices = { -1, -1, 1, 1, -1, 1, -1, 1, 1, 1, 1, 1, -1, -1, -1, 1, -1, -1, -1, 1, -1, 1, 1, -1};
// A color for each corner as an (R,G,B) tripletstatic byte[] cubeColors = { (byte) 255, (byte) 0, (byte) 0, // red … (byte) 0, (byte) 255, (byte) 0, // green};
// Define the cube as a single triangle stripstatic int[] indices = { 6,7,4,5,1,7,3,6,2,4,0,1,2,3 };static int[] stripLen = { 14 };
Define raw data for a cube
| 2004 JavaOneSM Conference | Session TS-2121 29
Example: Rotating cube
// Fill in VertexArrays from short[] and byte[]int numVertices = vertices.length/3;VertexArray pos = new VertexArray(numVertices, 3, 2);VertexArray col = new VertexArray(numVertices, 3, 1);pos.set(0, numVertices, cubeVertices);col.set(0, numVertices, cubeColors);
// Attach the VertexArrays to a VertexBuffer// Note the scale (1.0) and bias (0,0,0)vertices = new VertexBuffer();vertices.setPositions(pos, 1.0f, null);vertices.setColors(col);
// Fill in the triangle striptriangles = new TriangleStripArray(cubeIndices, stripLen);
// Create a Mesh with default Appearancecube = new Mesh(vertices, triangles, new Appearance());
Copy raw data into Objects
| 2004 JavaOneSM Conference | Session TS-2121 30
Example: Rotating cube
Camera cam = new Camera();
// 60-degree field of view, screen aspect ratio// Near clipping plane at 1.0, far plane at 1000
float aspect = (float) getWidth() / (float) getHeight();cam.setPerspective(60.0f, aspect, 1.0f, 1000.0f);
// Place the camera at z=5.0 in world space// View vector is along the negative Z axis
transform = new Transform();transform.postTranslate(0.0f, 0.0f, 5.0f);
g3d = Graphics3D.getInstance();g3d.setCamera(cam, transform);
Set up a Camera
| 2004 JavaOneSM Conference | Session TS-2121 31
Example: Rotating cube
g3d = Graphics3D.getInstance();g3d.bindTarget(getGraphics());
try { g3d.clear(null); g3d.render(cube, transform);} finally { g3d.releaseTarget(); }
xAngle += 1; yAngle += 2; zAngle += 3;transform.setIdentity();transform.postRotate(xAngle, 1.0f, 0.0f, 0.0f);transform.postRotate(yAngle, 0.0f, 1.0f, 0.0f);transform.postRotate(zAngle, 0.0f, 0.0f, 1.0f);
Clear the buffers, render, animate
| 2004 JavaOneSM Conference | Session TS-212132
Demo
Rotating cube
32
| 2004 JavaOneSM Conference | Session TS-2121 33
Agenda
Why should I care?
Getting started
Low level features
Bitmaps and texturing
Scene graph
Dynamic meshes
Animation
| 2004 JavaOneSM Conference | Session TS-2121 34
Add bitmap graphics
• 3D graphics is mostly vector graphics─ Mathematical shape definitions─ Define camera, lights, simulate─ “Computer graphics look”, pretty boring
• Bitmap graphics adds visual richness─ Background images─ Texture maps on surfaces─ Sprites
| 2004 JavaOneSM Conference | Session TS-2121 35
Texturing tricks
• Tip: Use light maps for lighting effects─ Usually faster than per-vertex lighting─ Use luminance textures, not RGB─ Multitexturing is your friend
• Tip: Use the perspective correction hint─ Almost always faster than adding more vertices─ But first check if the implementation supports it!
| 2004 JavaOneSM Conference | Session TS-2121 36
More bitmap tricks
• Tip: Use background images ─ Can be scaled, tiled and scrolled very flexibly─ Generally much faster than sky boxes or similar
• Tip: Use sprites as impostors, particles─ Generally (much) faster than textured quads─ Unscaled mode is (much) faster than scaled
• Tip: Load your images with the Loader─ Loader.load(“/img.png”) outputs Image2D─ Going through MIDP Image is a waste of memory
| 2004 JavaOneSM Conference | Session TS-212137
Demo
Backgrounds and texturing
37
| 2004 JavaOneSM Conference | Session TS-2121 38
Agenda
Why should I care?
Getting started
Low level features
Bitmaps and texturing
Scene graph
Dynamic meshes
Animation
| 2004 JavaOneSM Conference | Session TS-2121 39
The scene graph
SkinnedMesh
Group
Group
Group
Mesh
Sprite
Light
World
Group Camera
Group MorphingMesh
Actually, it’s just a tree
Not allowed!
| 2004 JavaOneSM Conference | Session TS-2121 40
VertexArray
Background
Image2D
VertexBuffer
IndexBuffer
Appearance
Image2D
Texture2D
MeshGroup
Mesh Mesh
Group Light Camera
World
How to find an object in the scene
0
12 3 4
4
5 6
1
0
6
7 89 10
11
UserID defaults to 0
UserIDs not necessarily unique
find(4)
Diagram courtesy of Sean Ellis, Superscape
Mesh4
| 2004 JavaOneSM Conference | Session TS-2121 41
• From this node to the parent node─ Ignored for the root node
• Composed of four parts─ Translation T─ Orientation R─ Non-uniform scale S─ Generic 3x4 matrix M
• Composite: C = T R S M
Node transformations
Group
Group
Group
Mesh
Sprite
C
CC
C C
WorldC
| 2004 JavaOneSM Conference | Session TS-2121 42
Node transformations
• Tip: Keep the transformations simple─ Favor the T R S components over M─ Avoid non-uniform scales in S
• Tip: How to scale along an arbitrary axis─ There is no direct support (C = T R N-1 S N M)─ Method 1: R’ = R N-1 and M’ = N M─ Method 2: Use an extra Group node
• Tip: How to rotate about an arbitrary point─ Again, no direct support (C = T P-1 R P S M)─ Method 1: Use an extra Group node─ Method 2: If S is identity, combine into T and M
| 2004 JavaOneSM Conference | Session TS-2121 43
Example: Set up a hierarchy
private Group group1, group2;private Mesh cube1, cube2, cube3;…cube1 = new Mesh(cubeVB, triangles, appearance);cube2 = new Mesh(cubeVB, triangles, appearance);cube3 = new Mesh(cubeVB, triangles, appearance);
group1 = new Group();group2 = new Group();
group1.addChild(cube1);group1.addChild(cube2);group1.addChild(group2);group2.addChild(cube3);
group2
group1
cube1
cube3
cube2
| 2004 JavaOneSM Conference | Session TS-2121 44
Animate and render the hierarchy
g3d.render(group1, null);…cube1.setOrientation( yAngle, 0.0f, 1.0f, 0.0f );cube1.setTranslation( 0.0f, 2.0f, 0.0f );
cube2.setOrientation(-yAngle, 0.0f, 1.0f, 0.0f );cube2.setTranslation( 0.0f,-2.0f, 0.0f );
group2.setOrientation(-yAngle, 0.0f, 1.0f, 0.0f );group2.setTranslation( -2.0f, 0.0f, 0.0f );
cube3.setOrientation( zAngle, 0.0f, 0.0f, 1.0f );cube3.setTranslation( -2.0f, 0.0f, 0.0f );cube3.setScale( 0.5f, 0.5f, 0.5f );
| 2004 JavaOneSM Conference | Session TS-212145
Demo
Hierarchic transformations
45
| 2004 JavaOneSM Conference | Session TS-2121 46
Agenda
Why should I care?
Getting started
Low level features
Bitmaps and texturing
Scene graph
Dynamic meshes
Animation
| 2004 JavaOneSM Conference | Session TS-2121 47
MorphingMesh
• A base mesh and several morph targets
• Result = weighted sum of morph deltas
• Can morph any vertex attribute─ Vertex positions─ Colors─ Normals─ Texture coordinates
i
iiw BTBR
For vertex morphing animation
| 2004 JavaOneSM Conference | Session TS-2121 48
MorphingMesh
Base Target 1eyes closed
Target 2mouth closed
Animate eyesand mouth
independently
Example: Animating a rabbit’s face
| 2004 JavaOneSM Conference | Session TS-2121 49
SkinnedMesh
i
iii vwv BM'
Articulated characters without cracks at joints
• Stretch a mesh over a hierarchic “skeleton”─ The skeleton consists of scene graph nodes─ Each node (“bone”) defines a transformation─ Each vertex is linked to one or more bones
| 2004 JavaOneSM Conference | Session TS-2121 50
SkinnedMesh
Neutral pose, bones at rest
Bone BBone A
"skin"shared vertex,
weights = (0.5, 0.5)non-sharedvertex
Weighted skinning
| 2004 JavaOneSM Conference | Session TS-2121 51
SkinnedMesh
Bone B rotated 90 degrees
Weighted skinning
| 2004 JavaOneSM Conference | Session TS-2121 52
SkinnedMesh
No skinning Local skinningone bone per vertex
Smooth skinningtwo bones per vertex
Example: Animating an arm
| 2004 JavaOneSM Conference | Session TS-2121 53
Agenda
Why should I care?
Getting started
Low level features
Bitmaps and texturing
Scene graph
Dynamic meshes
Animation
| 2004 JavaOneSM Conference | Session TS-2121 54
Animation: Relevant classes
KeyframeSequence
AnimationController
AnimationTrack
A link between sequence, controller and target
Object3D
Base class for all objects that can be animated
Controls the playback of one or more animations
Storage for keyframesDefines interpolation mode
| 2004 JavaOneSM Conference | Session TS-2121 55
KeyframeSequence
Keyframe is a time and the value of a property at that time
Can store any number of keyframes
Several keyframe interpolation modes
Can be open or closed (looping)
sequence timet
v
KeyframeSequence
Diagram courtesy of Sean Ellis, Superscape
| 2004 JavaOneSM Conference | Session TS-2121 56
AnimationController
Can control several keyframed animations together
Determines relationship between world time and sequence time
0 dsequence time
world timet
0
t
s
0 dsequence time0 dsequence time
AnimationController
Diagram courtesy of Sean Ellis, Superscape
| 2004 JavaOneSM Conference | Session TS-2121 57
Animation
Relates animation controller, keyframe sequence, and object property together.
0 dsequence time
Identifies animated property on this object
Call toanimate(worldTime)
sv
Calculate sequence time from world time
Look up value at this sequence time
Apply value to animated property
Diagram courtesy of Sean Ellis, Superscape
Object3D
AnimationTrack
AnimationController
KeyframeSequence
| 2004 JavaOneSM Conference | Session TS-2121 58
Animation
• Tip: You can read back the animated values─ Use the animation engine for anything you want─ Much faster than doing interpolation in Java─ Especially in case of quaternions and splines
| 2004 JavaOneSM Conference | Session TS-2121 59
Example: Set up keyframes
KeyframeSequence createKeyframeSequence() {
// 10 keys, 3 components, spline interpolation KeyframeSequence ks; ks = new KeyframeSequence(10,3,KeyframeSequence.SPLINE);
// x grows linearly, y wiggles up-down, z remains 0.0f float[] tb = { 0.0f, 0.0f, 0.0f }; tb[0] = -4.0f; tb[1] = 0.0f; ks.setKeyframe(0, 0, tb); tb[0] = -3.0f; tb[1] = 2.0f; ks.setKeyframe(1, 10, tb); … tb[0] = 5.0f; tb[1] = 0.0f; ks.setKeyframe(9, 90, tb);
ks.setDuration(100); // sequence duration in local time ks.setValidRange(0, 9); // all 10 keyframes are valid ks.setRepeatMode(KeyframeSequence.LOOP); return ks;}
| 2004 JavaOneSM Conference | Session TS-2121 60
Example: Set up controller
controller = new AnimationController();
// The animation is active between these world timescontroller.setActiveInterval(0, 1500);
// Set speed, scale “around” given time (0)controller.setSpeed(2.0f, 0);
// Create animation track with keyframes & controllerks = createKeyframeSequence();at = new AnimationTrack(ks,AnimationTrack.TRANSLATION); at.setController(controller);
// Attach it to our cubecube.addAnimationTrack(at);
| 2004 JavaOneSM Conference | Session TS-212161
Demo
Keyframe animation
61
| 2004 JavaOneSM Conference | Session TS-2121 62
Summary
• M3G is the 3D API for J2ME™ technology─ Will be supported in millions of devices
• Not just an API─ M3G also defines a binary file format
• M3G is flexible─ Using the immediate mode gives you full control─ High-level features make your code faster and smaller
| 2004 JavaOneSM Conference | Session TS-212163
Demo
3D snowboarding
63
| 2004 JavaOneSM Conference | Session TS-2121 64
For More Information
• M3G specificationwww.forum.nokia.com/java/jsr184
• SDKs, additional documentationwww.forum.nokia.com/ (click “Series 40”)developer.superscape.com/ (registration needed)
• M3G content creation toolswww.discreet.com/www.superscape.com/
| 2004 JavaOneSM Conference | Session TS-212165
Q&A
Thanks: Kari Kangas, Sean Ellis, Nokia M3G team, JSR-184 Expert Group
65
java.sun.com/javaone/sf
| 2004 JavaOneSM Conference | Session TS-212166
Advanced Game Development with the Mobile 3D Graphics API
Tomi Aarnio, Kari PulliNokia Research Center