1.developing web graphics with WebGL tony parisi October 22, 2013

2. about me serial entrepreneur founder, stealth startup consulting architect and CTO co-creator, VRML and X3D web standards co-designer, glTF author, speaker instructor contact information tparisi@gmail.com skype: auradeluxe http://twitter.com/auradeluxe p://www.tonyparisi.com/ http://www.learningwebgl.com/book source code https://github.com/tparisi/WebGLBook https://github.com/tparisi/Programming3DApplications httget the books! WebGL: Up and Running http://www.amazon.com/dp/144932357X Programming 3D Applications with HTML and WebGL http://www.amazon.com/Programming-ApplicationsHTML5-WebGL-Visualization/dp/1449362966http://www.tonyparisi.com10/22/2 013 3. RIP: 1995-2013Image: Eric Dyehttp://www.tonyparisi.com10/22/2 013 4. long livehttp://www.tonyparisi.com10/22/2 013 5. WebGL: real-time 3D rendering the 3D API standard OpenGL ES in a browser JavaScript API bindings supported in all modern browsers shipped since early 2011 supported in desktop Safari, Firefox, Chrome, Opera Internet Explorer (late 2013) iOS mobile Safari iAds only Android mobile Chrome, mobile Firefox Blackberry, Tizen, Firefox OSSurface (Windows 8.1) Kindle Fire HDX 500M+ seats -> 1B http://www.tonyparisi.com10/22/2 013 6. science and education100,000 Stars Google Experiment http://workshop.chromeexperiments.com/stars/http://www.tonyparisi.com10/22/2 013 7. advertising and mediacollaboration with Rei Inamoto and AKQA http://makeourmark.levi.com/project-overview-whatmovesyou.html developed by Tony Parisi and Simo Santavirta http://www.simppa.fi/ http://www.tonyparisi.com10/22/2 013 8. data visualizationhttp://www.tonyparisi.com10/22/2 013 9. page graphicsSteve Wittens http://acko.net/blog/making-mathbox/ http://www.tonyparisi.com10/22/2 013 10. games and virtual environments60FPSported in 5 days Unreal native C++ engine -> JavaScriptEmscripten + asm.js http://www.tonyparisi.com10/22/2 013 11. products and e-commerceproduct concept piece Vizi - TC Chang/T. Parisi http://vizi.gl/engine/tests/futurgo.html http://www.tonyparisi.com10/22/2 013 12. how WebGL works its a JavaScript drawing API draw to a canvas element using a special context (webgl) low-level drawing buffers, primitives, textures and shaders accelerated by graphics hardware (GPU) can draw 2D as well as 3D graphicsintegrates seamlessly with other page content there is no file format; no markup language; no DOM. libraries and frameworks are key to fast ramp up and productive developmenthttp://www.tonyparisi.com10/22/2 013 13. a simple WebGL program 1. create a element 2. obtain a drawing context 3. initialize the viewport4. create one or more buffers 5. create one or more matrices 6. create one or more shaders7. initialize the shaders 8. draw one or more primitiveshttp://www.tonyparisi.com10/22/2 013 14. create the canvas, context and viewport function initWebGL(canvas) { var gl = null; var msg = "Your browser does not support WebGL, " + "or it is not enabled by default."; try { gl = canvas.getContext(webgl"); } catch (e) { msg = "Error creating WebGL Context!: " + e.toString(); }detect WebGLif (!gl) { alert(msg); throw new Error(msg); } return gl; } function initViewport(gl, canvas) { gl.viewport(0, 0, canvas.width, canvas.height); }set WebGL drawing regionhttp://www.tonyparisi.com10/22/2 013 15. buffers and typed arrays var vertexBuffer; vertexBuffer = gl.createBuffer(); gl.bindBuffer(gl.ARRAY_BUFFER, vertexBuffer); var verts = [ // Front face -1.0, -1.0, 1.0, 1.0, -1.0, 1.0, 1.0, 1.0, 1.0, -1.0, 1.0, 1.0, // Back face -1.0, -1.0, -1.0, -1.0, 1.0, -1.0, 1.0, 1.0, -1.0, 1.0, -1.0, -1.0, WebGL drawing functions use buffers of datanew low-level data type stores arrays of floats and ints compactly]; gl.bufferData(gl.ARRAY_BUFFER, new Float32Array(verts), gl.STATIC_DRAW); http://www.tonyparisi.com10/22/2 013 16. shaders var vertexShaderSource = " " " " " " " " " " " "the vertex shader attribute vec3 vertexPos;n" + attribute vec2 texCoord;n" + transforms model-space uniform mat4 modelViewMatrix;n" + positions into screen uniform mat4 projectionMatrix;n" + space varying vec2 vTexCoord;n" + void main(void) {n" + // Return the transformed and projected vertex valuen" + gl_Position = projectionMatrix * modelViewMatrix * n" + vec4(vertexPos, 1.0);n" + // Output the texture coordinate in vTexCoordn" + vTexCoord = texCoord;n" + }n";the fragment shader var fragmentShaderSource = " precision mediump float;n" + outputs a color value " varying vec2 vTexCoord;n" + for each pixel " uniform sampler2D uSampler;n" + " void main(void) {n" + " // Return the pixel color: always output whiten" + " gl_FragColor = texture2D(uSampler, vec2(vTexCoord.s, vTexCoord.t));n" + "}n"; http://www.tonyparisi.com10/22/2 013 17. drawing function draw(gl, obj) { // clear the background (with black) gl.clearColor(0.0, 0.0, 0.0, 1.0); gl.enable(gl.DEPTH_TEST); gl.clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT);clear the canvas// set the shader to use gl.useProgram(shaderProgram);set the shader// connect up the shader parameters: vertex position, texture coordinate, // projection/model matrices and texture // set up the buffers gl.bindBuffer(gl.ARRAY_BUFFER, obj.buffer); gl.vertexAttribPointer(shaderVertexPositionAttribute, obj.vertSize, gl.FLOAT, false, 0, 0); gl.bindBuffer(gl.ARRAY_BUFFER, obj.texCoordBuffer); gl.vertexAttribPointer(shaderTexCoordAttribute, obj.texCoordSize, gl.FLOAT, false, 0, 0); gl.bindBuffer(gl.ELEMENT_ARRAY_BUFFER, obj.indices); gl.uniformMatrix4fv(shaderProjectionMatrixUniform, false, projectionMatrix); gl.uniformMatrix4fv(shaderModelViewMatrixUniform, false, modelViewMatrix); gl.activeTexture(gl.TEXTURE0); gl.bindTexture(gl.TEXTURE_2D, webGLTexture); gl.uniform1i(shaderSamplerUniform, 0); // draw the object gl.drawElements(obj.primtype, obj.nIndices, gl.UNSIGNED_SHORT, 0); }set up buffers for vertices and texture coordinates pass transform and projection matrices to the shader set the texture and pass to the shader draw the objecthttp://www.tonyparisi.com10/22/2 013 18. a WebGL client-side stack rendering Three.js library animation Three.js, Tween.js libraries application functionality Vizi framework setup/teardown interaction picking, rollovers, rotating models behaviors run loop and event dispatchingcontent creation pipeline 3D tools e.g. Autodesk Maya COLLADA2GLTF converter Vizi web-based previewerhttp://www.tonyparisi.com10/22/2 013 19. three.js: a JavaScript 3D engine https://github.com/mrdoob/three.js/the most popular WebGL library renderer = new THREE.WebGLRenderer( { canvas: canvas, antialias: true } ); scene = new THREE.Scene(); camera = new THREE.PerspectiveCamera( 45, canvas.width / canvas.height, 1, 4000 ); scene.add(camera); var light = new THREE.DirectionalLight( 0xffffff, 1.5); scene.add( light ); var mapUrl ="../images/webgl-logo-256.jpg; var map = THREE.ImageUtils.loadTexture(mapUrl ); var material = new THREE.MeshPhongMaterial({ map: map }); var geometry = new THREE.CubeGeometry(2, 2, 2); cube = new THREE.Mesh(geometry, material); scene.add( cube );can render to WebGL, 2D canvas, SVG and CSS3 represents WebGL at a high level using standard 3D graphics concepts http://www.tonyparisi.com10/22/2 013 20. 3D animation requestAnimationFrame() http://www.paulirish.com/2011/requestanimationframe-forsmart-animating/with JavaScript we can animate anything: materials, lights, textures. Shaders Three.js has support for key frames, morphs and skinsTween.js simple tweening library https://github.com/sole/tween.js/ var tween = new TWEEN.Tween( { x: 50, y: 0 } ) .to( { x: 400 }, 2000 ) .easing( TWEEN.Easing.Elastic.InOut ) .start(); function animate() { requestAnimationFrame( animate ); TWEEN.update(); }create and run tween call TWEEN.update() each frame to update values http://www.tonyparisi.com10/22/2 013 21. creating the car (site) of the futurehttp://www.tonyparisi.com10/22/2 013 22. the conceptdesign: TC Chang (http://www.tcchang.com/) http://www.tonyparisi.com10/22/2 013 23. the modelhttp://www.tonyparisi.com10/22/2 013 24. the content pipeline - today still pretty crude tools and converters under development sample work flows A.OBJ (single model) 1. 2. 3.create 3D model or import into Blender export to Three.js JSON format load into Three.js application; hand-layout, hand-light, hand-animateB. OBJ (single model) 1. 2.convert to Three.js JSON using Python command-line tool load into Three.js application; hand-layout, hand-light, hand-animateC. MD2/MD5 (Quake single model with animation) 1. 2.convert to Three.js JSON with online tool load into Three.js application; hand-layout, hand-lightD. COLLADA (full scene) 1. 2. 3. 4.export to COLLADA from Maya, 3ds Max, Blender, Sketchup files contain scene layout, cameras, lights and animations no need to do it by hand import into Three.js application and use but COLLADA files are big to download and slow to parse http://www.tonyparisi.com10/22/2 013 25. the content pipeline - coming soon: glTF a JPEG for 3D bridges the gap between existing 3D formats/tools and todays GL based APIs compact, efficient to load representation balanced, pragmatic feature set GL native data types require no additional processing also includes common 3D constructs (hierarchy, cameras, lights, common materials, animation )reduces duplicated effort in content pipeline a common publishing format for content tools open specification; open process Khronos Group initiative within the COLLADA working group F. Robinet (lead), R. Arnaud, P. Cozzi, T. Parisi http://gltf.gl/ http://www.tonyparisi.com10/22/2 013 26. exporting and previewing content tree view of 3D scene graphproperty sheets show detailscene/render statscameras, lights and animations http://www.tonyparisi.com10/22/2 013 27. Vizi: a framework for 3D applications component-based framework a la Unity3D Futurgo.prototype.go = function() { this.viewer = new Vizi.Viewer({ container : this.container, showGrid : true, allowPan: false, oneButton: true }); this.loadURL(Futurgo.URL); built-in Viewer object for this.viewer.run(); viewing models handles } Futurgo.prototype.loadURL = function(url) { var that = this; var loader = new Vizi.Loader; loader.addEventListener("loaded", function(data) { that.onLoadComplete(data, loadStartTime); }); var loadStartTime = Date.now(); loader.loadScene(url); } Futurgo.prototype.onLoadComplete = function(data, loadStartTime) { var scene = data.scene; this.viewer.replaceScene(data); Three.js setup, page events (e.g. resize), mouse events, run loop built-in Loader object loads models in Three.js JSON format, COLLADA, glTF add newly loaded model to scene http://www.tonyparisi.com10/22/2 013 28. interaction and behaviors API methods use regular expression or object type to iterate over scene graph// Add entry fade behavior to the windows var that = this; scene.map(/windows_front|windows_rear/, function(o) { var fader = new Vizi.FadeBehavior({duration:2, opacity:.8}); o.addComponent(fader); setTimeout(function() { component model fader.start(); drop behaviors onto }, 2000);any object var picker = new Vizi.Picker; picker.addEventListener("mouseover", function(event) { that.onMouseOver("glass", event); }); picker.addEventListener("mouseout", function(event) { that.onMouseOut("glass", event); }); picker.addEventListener("touchstart", function(event) { that.onTouchStart("glass", event); }); picker.addEventListener("touchend", function(event) { pickers - dispatch mouse and that.onTouchEnd("glass", event); }); touch events to application o.addComponent(picker); 10/22/2 }); http://www.tonyparisi.com 013 29. animations Vizi Viewer object maintains list of named animations Futurgo.prototype.playAnimation = function(name, loop, reverse) { var animationNames = this.viewer.keyFrameAnimatorNames; loaded from var index = animationNames.indexOf(name); content file (e.g. if (index >= 0) COLLADA, glTF) { this.viewer.playAnimation(index, loop, reverse); } } Futurgo.prototype.playOpenAnimations = function() { this.playAnimation("animation_window_rear_open"); this.playAnimation("animation_window_front_open"); }play animations looped or unlooped, forward or reverseFuturgo.prototype.playCloseAnimations = function() { this.playAnimation("animation_window_rear_open", false, true); this.playAnimation("animation_window_front_open", false, true); } http://www.tonyparisi.com10/22/2 013 30. integrating 2D and 3D 3D rollovers trigger 2D overlay display2D page elements control 3D scene content http://www.tonyparisi.com10/22/2 013 31. links game engines/IDEslibraries/frameworksPlayCanvas http://www.playcanvas.com/Three.jsTurbulenz https://turbulenz.com/SceneJSGoo Enginehttp://www.gootechnologies.c om/http://threejs.org/http://scenejs.org/BabylonJS http://www.babylonjs.com/Artillery Engine https://artillery.com/Voodoo.js http://www.voodoojs.com/Verold http://verold.com/ tQuery Sketchfab https://sketchfab.com/http://jeromeetienne.github.io/tquery/Unreal ?Vizi (VERY preliminary)http://www.extremetech.com/gaming/15190 0-unreal-engine-3-ported-to-javascript-andwebgl-works-in-any-modern-browserhttps://github.com/tparisi/Vizihttp://www.tonyparisi.com10/22/2 013 32. stay in touch enjoy the show! http://html5devconf.com/contact information tparisi@gmail.com skype: auradeluxe http://twitter.com/auradeluxe p://www.tonyparisi.com/ http://www.learningwebgl.com/book source code https://github.com/tparisi/WebGLBook https://github.com/tparisi/Programming3DApplications httget the books! WebGL: Up and Running http://www.amazon.com/dp/144932357X Programming 3D Applications with HTML and WebGL http://www.amazon.com/Programming-ApplicationsHTML5-WebGL-Visualization/dp/1449362966http://www.tonyparisi.com10/22/2 013