© Copyright Khronos Group 2015 - Page 1
The state of WebGL and glTFPatrick Cozzi | @pjcozzi
3D Formats Working Group
September 2015
© Copyright Khronos Group 2015 - Page 2
Khronos Connects Software to Silicon
Open Consortium creating
ROYALTY-FREE, OPEN STANDARD
APIs for hardware acceleration
Defining the roadmap for
low-level silicon interfaces needed
on every platform
Graphics, compute
and vision processing
Rigorous specifications AND
conformance tests for cross-vendor
portability
Acceleration APIs
BY the Industry
FOR the Industry
Well over a BILLION people use Khronos APIs
Every Day…
© Copyright Khronos Group 2015 - Page 3
Today’s Topics• What’s new? What’s upcoming?
- Spec and ecosystem
Today’s Main Focus
© Copyright Khronos Group 2015 - Page 4
A Few Recent WebGL Apps
Red Bull X-Alps
NASA Curiosity RoverMarmoset Viewer
© Copyright Khronos Group 2015 - Page 5
Strong WebGL Ecosystem• Platforms
- Desktop, Android, and iOS
• Engines
- Three.js, BabylonJS, SceneJS, xeoEngine, Pex, Blend4Web, Cesium
• Tools
- Browser debuggers/profilers, Shader Editors
WebGL Inspector, Web Tracing Framework
• Education
- Books, online courses, and tutorials
• Adoption across many domains, not just games
© Copyright Khronos Group 2015 - Page 6
WebGL 2.0• Brings most OpenGL ES 3.0 features to WebGL to enable
- Better visual quality
- Better performance
- New GPU algorithms
• Draft spec is available
- https://www.khronos.org/registry/webgl/specs/latest/2.0/
• Prototype implementations in Chrome and Firefox
- https://www.khronos.org/webgl/wiki/Getting_a_WebGL_Implementation
• Chromium aiming to pass all known conformance tests by the end of the year
• Many WebGL 2.0 features are available today as extensions
- ANGLE_instanced_arrays, OES_vertex_array_object, WEBGL_draw_buffers, ...
© Copyright Khronos Group 2015 - Page 7
Major WebGL 2.0 Features• WEBGL_draw_buffers (Multiple Render Targets)
- Deferred shading, Order-Independent Transparency (OIT), …
© Copyright Khronos Group 2015 - Page 8
Major WebGL 2.0 Features• Instancing
- Less draw calls to draw lots of similar objects (instances)
© Copyright Khronos Group 2015 - Page 9
Major WebGL 2.0 Features• Uniform buffers
• Transform feedback
• Multisampled Renderbuffers
• 3D textures
• NPOT textures
• More texture formats
• Occlusion queries
• Vertex array objects
• Sampler objects
• Sync objects
• Fragment depth
• Primitive restart
• …
© Copyright Khronos Group 2015 - Page 10
Beyond graphics APIs,
we need a runtime
3D Model Format
10
© Copyright Khronos Group 2015 - Page 11
What’s in a 3D model?
Node hierarchy and geometry
Materials and textures
Animations and skins
© Copyright Khronos Group 2015 - Page 12
Native Modeling Tool Format Examples
12
.lxo.blend .ma / .mb
© Copyright Khronos Group 2015 - Page 13
Interchange Formats
…
.fbx
.obj
…
© Copyright Khronos Group 2015 - Page 14
Interchange Formats
.fbx
Engine(Runtime)
.obj
??
© Copyright Khronos Group 2015 - Page 15
Interchange Formats
•Target tools - not the graphics API
•Example: COLLADA- XML + image files
- One index per attribute, not vertex
- Unsigned int indices
- Transform stack per node
- Polygons and splines
- Doesn’t specify image file format
- Lots of flexibility and indirection in animations and skins
15
© Copyright Khronos Group 2015 - Page 16
Runtime Format• Optimized for use in a runtime engine
Interchange
format
Tool
format
Content
Pipeline
Runtime
formatEngine
© Copyright Khronos Group 2015 - Page 17
3D Needs a Transmission Format!• Bridge the gap between tools and ‘GL’ based apps
- Reduce duplicated effort in content pipelines
- Enable richer 3D representation – OBJ, STL etc. too limited
- Provide common publishing format for content tools and services
Audio Video Images 3D
MP3 H.264 JPEG ?!
A widely adopted format ignites previously
unimagined opportunities for a media type
© Copyright Khronos Group 2015 - Page 18
glTF = “JPEG for 3D”• ‘GL Transmission Format’
- Runtime asset format for WebGL, OpenGL ES, and OpenGL applications
• Compact representation for download efficiency
- Binary mesh and animation data
• Loads quickly into memory
- GL native data types require no additional parsing
• Full-featured
- 3D constructs (node hierarchy, materials, animation, cameras, lights)
• Runtime Neutral
- Can be created and used by any tool, app, or runtime
• Flexible Extensibility
- E.g. payloads with compression and streaming
© Copyright Khronos Group 2015 - Page 19
glTF Internals
19
.json
Node hierarchy, materials, lights, cameras
.bin
• Geometry: vertices and
indices
• Animation: key-frames
• Skins: inverse-bind matrices
.glsl
Shaders
.png, .jpg, …
Textures
© Copyright Khronos Group 2015 - Page 20
scene
node
camera mesh light
accessor
bufferView
buffer
material
technique texture
samplerimageprogram
shader
1
2
11
1
1 *
**
*11
*
*
*
skin
animation
*
1
1
glTF Internals
© Copyright Khronos Group 2015 - Page 21
glTF Internals
© Copyright Khronos Group 2015 - Page 22
glTF Project Status• Open specification; Open process
- Specification and multiple loaders and translators in open source
- https://github.com/KhronosGroup/glTF
• Draft glTF 1.0 spec is imminent
- spec-1.0 branch
• Extension mechanisms fully defined
- Vendor, multi-vendor and official Khronos extensions (mirrors OpenGL)
- Anyone can ship vendor extensions at any time – no permissions needed
- First extensions will be included in launch
http://www.tonyparisi.com
© Copyright Khronos Group 2015 - Page 23
glTF Adoptionthree.js Loader
https://github.com/mrdoob/three.js/It’s the native format!
http://cesiumjs.org/
Babylon.js Loader (in development)
http://www.babylonjs.com/
collada2gltf converterhttps://github.com/KhronosGroup/glTF
FBX to glTF Convertor(in development)
Drag and drop convertor coming
http://gltf.autodesk.io/
Online drag and drop COLLADA
to glTF converterhttp://cesiumjs.org/convertmodel.html
PIPELINE TOOLS
Native import and
display of glTF models
3D Advertising Solutions
with native glTF import
© Copyright Khronos Group 2015 - Page 24
Initial glTF Extensions• KHR_binary_glTF (Khronos extension)
- Enables a glTF file to refer to external binary asset packages
• EXT_quantized_attributes (vendor extension)
- Quantization-based attribute compression
- Decompression in vertex shader
• Open3DGC – MPEG 3D mesh compression (in progress)
- C++ encoder/decoder + JavaScript decoder
- 40–80% compression
- Extensions inserts decompression between file buffer and vertex data
- Building support into COLLADA2GLTF converter and Cesium loader
© Copyright Khronos Group 2015 - Page 25
Open3DGC glTF Extension Initial Results
ModelVertices Tris Flat +
Gzip
Open3DGC +
Gzip
Compression
AmountJavaScript
Execution Time
COLLADA Duck 2.1k 4.2k 54 KiB 14 KiB -74% 24 ms
Stanford Bunny 2.5k 5.0k 105 KiB 56 KiB -47% 30 ms
Stanford Dragon 435k 871k 7792 KiB 2141 KiB -73% 630 ms
3D Tile 12.8k 6.5k 102 KiB 59 KiB -42% —
OpenStreetMap NYC — — 337 MiB 207 MiB -39% (Streamed)
Google Chrome 44.0, Windows 8.1, Intel i7-4980HQ @ 2.80GHz
© Copyright Khronos Group 2015 - Page 26
Cesium 3D Tiles Using glTF• An open specification for streaming massive 3D geospatial datasets
- Streams 3D content including buildings, trees, point clouds, and vector data
• Hierarchical Level of Detail (HLOD)
- Only visible and prioritized tiles are streamed
- glTF payloads can be compressed, e.g., using 3DGC extension
Over 1.1 million OpenStreetMap buildings in New York City.
© Copyright Khronos Group 2015 - Page 27
Get Involved!• WebGL
- Experiment with WebGL 2 implementations:
https://www.khronos.org/webgl/wiki/Getting_a_WebGL_Implementation
• glTF
- Review the spec:
https://github.com/KhronosGroup/glTF/blob/spec-1.0/specification/README.md
• Contact
- Patrick Cozzi
- @pjcozzi