developing interactive 3d experiences in html5 with carlos ulloa
DESCRIPTION
Presented live at FITC Amsterdam 2014 on Feb 24-25, 2014 More details can be found at www.FITC.ca Developing Interactive 3D Experiences in HTML5 with Carlos Ulloa Modern web browsers are capable of producing some amazing audiovisual candy, through new capabilities like HTML5, fast JavaScript, WebGL and Web Audio API. In this session, Carlos talks in detail about his work developing interactive 3D experiences using these technologies. He will take an in-depth look at the production of high-end WebGL projects including Lights, HelloRun and Racer-S, to discuss the graphics, audio and input development techniques used. He shows how to take advantage of the three.js engine to deliver immersive 3D visuals running at high performance on desktop and mobile browsers. This presentation will also cover music visualization and sound effects, using Web Audio API for sound processing and analysis. Carlos also discusses the integration of a broad range of input technologies like touch, mouse, keyboard, accelerometer and Leap Motion controller, along with some interesting lessons learned.TRANSCRIPT
Developin! Interactive3D Experiences in HTML5
Carlos UlloaHelloEnjoy.com
FITC Amsterdam 2014
C4RL05@
AUDIOGFX INPUT
Developin!Interactive 3D Experiences inHTML5
1-CLICK
WORKFLOW
Workflow first& then performance
Test earlyFirst build in 2 weeks
Test oftenOne or two builds per week
PrototypeFrom concept & design
GFX
GAME LOOP
Optimiseyour game loop
Object PoolingMinimise garbage collection
Spread& interpolate over frames
TIME
Frame RateIndependent
requestAnimationFrame
deltaTimeSince the last frame
Multiply by
deltaTimeto update values
per secondinstead of per frame
Cap deltaTime< 0.1 second
timeScaleMultiplies deltaTime
Keep Syncwith music & video
THREE.JS
Source Codethree.js/src
ASSET PIPELINE
MATH
QUATVEC MTRX
VECTORS
VectorsRepresent position, direction,
velocity, acceleration, forces…
( x, y, z )Not very intuitive on multiple axes
object.positionLocal position
object.scaleLocal scale
|a|Magnitude
|a| = 1Normalized
+
DOT PRODUCT
Dot Product
|a| |b| cosDot Product
CROSS PRODUCT
+
Cross Product
|a| |b| sinCross Product
QUATERNIONS
QuaternionsRepresent rotations
object.quaternionLocal rotation
Multiply quaternionsto rotate a vector or another rotation
Slerp quaternionsto interpolate rotations
Avoid Euleron multiple axes
MATRICES
MatricesRepresent transformationsi.e. position, rotation, scale
object.matrixLocal transform
object.matrixWorldGlobal transform
Multiply matricesto transform a vector or another matrix
Transform methodslookAt, translateXYZ, rotateXYZ,
localToWorld, worldToLocal
OBJECTS
HIERARCHY
GROUPS
PIVOTS
Children methodsgetDescendants, traverse
Helper ObjectsArrow, Axis, Bounding Box,
Camera, Grid, Lights
MESHES
GeometryBuild your own
Vertex ColorPaint, bake or create in code
TEXTURES
Mip MapsSmaller versions of the texture
Offset & RepeatTransform textures
AnisotropyFix mip map artifacts
WrapRepeat, Clamp, Mirror
Clamp WrapFix artifacts on texture seams
RGBAUse all channels
LIGHTS
Dynamic LightsReal-time surface illumination
Baked LightsBetter quality & performance
LightmapsGlobal illumination
Vertex BakeQuality depends on geometry
CAMERA
Avoiddefault camera settings
Near PlaneOcclusion & fill rate performance
Far PlaneHorizon pop up & removed geometry
FRUSTUMCULLING
Near Far
Boundin!Sphere
FOVVertical angle
FOV
FOVDifferent for each scene
POST PROCESSING
FXAA
8X
8X
SSAO
Cry En!ine
GLOW
VIGNETTE
OPTIMISE
Profile & measureUnderstand what’s going on
GPU CPU
GPU
GPU?Faster in lower res
GPUMany pixels, fill rate bound
CPU
CPU?Faster with objects disabled
CPUMany objects, many draw calls
DRAW CALLS
Minimisestate changes & draw calls
BatchingCombine meshes with same material
Batch StaticAlways combine static objects
Combine MaterialsUse vertex & textures to differentiate
Texture AtlasCombine textures
Multiple PassesMultiple draw calls
SHADERS
Objects < Vertices < Pixels
VertexInstead of fragment
PrecissionColors & normals
Alpha TestInstead of Alpha Blend
Fragment MathAvoid pow, sin, cos
Vector MathDot & Cross Product
LOD
Level of detailSimpler far away objects
Optimisedgeometries or materials
Sprites& particle systems
Skybox & Ringsfor very distant objects
MOBILE
Mobile < Desktop
Low Polycount> 500
OverdrawDepends on the GPU
Fill RatedevicePixelRatio
UpscaleFor higher resolutions
DownscaleFor higher quality renders
Simple ShadersMinimise fragment complexity
No BumpsAvoid normal mapping
No LightsAvoid dynamic lighting or shadows
No FogOnly on vertex shader
No PostAvoid full screen effects
lowpgetShaderPrecisionFormat
AUDIO
WEB AUDIO
AudioProcessing & analysis
Audio NodeWeb Audio API building block
Panner Nodefor positional audio
Logarithmicscale for spectrum analysis
INPUT
KEYBOARD
AnalogSimulate key pressure
Each keyon its own
Combinekeys to create axes
MOUSE
Pointer LockRelative mouse movement
Relative MouseAccumulated deltas
DPI& different screen sizes
SensitivityUsers ignore settings
TOUCH
Relativetouch controls
OrientationHow users hold the device?
Secondarytouch support
LEAP MOTION
SmoothIntroduces lag
CalibrationDon’t leave it to the user
RelativeReal time calibration
Slideshelloenjoy.com/docs/3DJSDev.pdf
