© The Khronos® Group Inc. 2018 - Page 1This work is licensed under a Creative Commons Attribution 4.0 International License

glTF: Efficient 3D Models

Tokyo, December 2018

© The Khronos® Group Inc. 2018 - Page 2This work is licensed under a Creative Commons Attribution 4.0 International License

Welcome to Khronos BOF Sessions!

Time Session Speakers

9:30AM-12:30PM 3D Graphics and Virtual Reality with Vulkan and OpenXR

Neil Trevett, NVIDIA - Introduction to Vulkan and OpenXRHai Nguyen, Google - Vulkan: Getting Started, Best Practices and using HLSL

Ashley Smith, AMD - Vulkan Memory and Heap Types – and When to Use Which!Nuno Subtil, NVIDIA - Using Subgroups to Bring More Parallelism to your Application

Teemu Virolainen, Basemark - Vulkan in the Rocksolid Engine

2:30PM-4:30PM WebGL: Latest Techniques

Xavier Ho, CSIRO’s Data61 and Juan Miguel de Joya, UN ITU - Welcome!Neil Trevett, NVIDIA - A Brief History of WebGL

Xavier Ho and Juan Miguel de Joya - Optimizing your WebGL Xavier Ho and Juan Miguel de Joya - Survey of WebGL Applications

Mike Alger and Germain Ruffle, Google - User Experiences for Digital/Augmented Spaces

4:30PM-5:30PM glTF: Efficient 3D Models Neil Trevett, NVIDIA - glTF Overview and Latest Updates Mark Callow, glTF Working Group - Universal Compressed Texture Transmission Format Update

5:30PM-5:35PM Japanese Greeting and Invitation to Khronos! Hitoshi Kasai, Khronos - Learn how your company can benefit from joining Khronos!

5:35PM-6PMStandards for machine learning,

inferencing and vision acceleration: NNEF, OpenVX and OpenCL

Neil Trevett, NVIDIA - Overview of NNEF, OpenVX and OpenCL and how they relate to each other to deliver effective acceleration for inferencing, machine learning and vision processing

All slides will be posted at www.khronos.org

© The Khronos® Group Inc. 2018 - Page 3This work is licensed under a Creative Commons Attribution 4.0 International License

Khronos Mission

Software

Silicon

Khronos is an International Industry Consortium creating royalty-free, open standards to enable software to access hardware acceleration for

3D Graphics, Virtual and Augmented Reality, Parallel Computing, Neural Networks and Vision Processing

Asian Members

© The Khronos® Group Inc. 2018 - Page 4This work is licensed under a Creative Commons Attribution 4.0 International License

Khronos Standards

High-performance, low-latency 3D Graphics

Download 3D object and scene data

Vision and sensor processing -including neural network

inferencing for machine learning

Portable interaction with VR/AR sensor, haptic and

display devices

© The Khronos® Group Inc. 2018 - Page 5This work is licensed under a Creative Commons Attribution 4.0 International License

glTF – The JPEG of 3D!

Efficient, reliable transmissionBring 3D assets into 1000s of

apps and engines

glTF 2.0 – June 2017Native AND Web APIsPhysically Based Rendering

Metallic-Roughness and Specular-Glossiness

glTF 1.0 – December 2015Primarily for WebGL

Uses GLSL for materials

glTF spec development on open GitHub – get involved!

https://github.com/KhronosGroup/glTF

Compact to TransmitSimple and Fast to LoadDescribes Full Scenes

Runtime NeutralOpen and Extensible

© The Khronos® Group Inc. 2018 - Page 6This work is licensed under a Creative Commons Attribution 4.0 International License

glTF 2.0• Graphics API neutral

- Proven by engine implementations using WebGL, Vulkan and Direct3D

• Physically Based Rendering (PBR) material definitions- Material information stored in textures

• Deployment as single file- Binary glTF (.glb) in core spec

• Morph Targets- Enhanced animation system

• Download the glTF 2.0 Reference Guide!- https://www.khronos.org/files/gltf20-reference-guide.pdf

© The Khronos® Group Inc. 2018 - Page 7This work is licensed under a Creative Commons Attribution 4.0 International License

glTF 2.0 Scene Description Structure

.gltf (JSON)Node hierarchy, PBR material textures, cameras

.binGeometry: vertices and indices

Animation: key-framesSkins: inverse-bind matrices

.png.jpg...

Textures

Texture based PBR materials

Geometry

Mandatory Metallic-Roughness Materials

Optional Specular-Glossiness Materials

© The Khronos® Group Inc. 2018 - Page 8This work is licensed under a Creative Commons Attribution 4.0 International License

glTF 2.0 Scene Description Structure

.gltf (JSON)Node hierarchy, PBR material textures, cameras

.binGeometry: vertices and indices

Animation: key-framesSkins: inverse-bind matrices

.png.jpg...

Textures

PBR materialsGeometry

© The Khronos® Group Inc. 2018 - Page 9This work is licensed under a Creative Commons Attribution 4.0 International License

PBR Materials

© The Khronos® Group Inc. 2018 - Page 10This work is licensed under a Creative Commons Attribution 4.0 International License

PBR Materials

© The Khronos® Group Inc. 2018 - Page 11This work is licensed under a Creative Commons Attribution 4.0 International License

PBR Materials

© The Khronos® Group Inc. 2018 - Page 12This work is licensed under a Creative Commons Attribution 4.0 International License

PBR Sample Applications

https://github.com/Microsoft/glTF-DXViewerhttp://github.khronos.org/glTF-WebGL-PBR/

© The Khronos® Group Inc. 2018 - Page 13This work is licensed under a Creative Commons Attribution 4.0 International License

glTF Binary - .glb• Binary version of glTF• Single file that encapsulates all geometry and textures• Ideal for end users to interact with a single file

© The Khronos® Group Inc. 2018 - Page 14This work is licensed under a Creative Commons Attribution 4.0 International License

Industry Support for glTF

https://www.khronos.org/gltf/#gltf_industry_support

© The Khronos® Group Inc. 2018 - Page 15This work is licensed under a Creative Commons Attribution 4.0 International License

glTF Ecosystem

Sony 3D Creator

Oculus

Paint 3D

Collada2gltf FBX2glTF

gltf-vscode

glTF-validator glTF-asset-generator

ExperienceApps and Engines

DiscoverRepositories

Drive DemandUsers

CreateTools

Windows Mixed Reality Home

Modo

Mixed Reality Viewer

3D BuilderPrep for 3D printing

OBJ2GLTF

glTF-Toolkit

Titania

HTC

glTF Reference Viewer

© The Khronos® Group Inc. 2018 - Page 16This work is licensed under a Creative Commons Attribution 4.0 International License

glTF Recent Highlights

Adds glTF to StemCell - 60K+ 3D artists and 700K 3D modelshttps://www.khronos.org/blog/turbosquid-adds-gltf-to-supported-formats-for-its-stemcell-initiative

Supports drag and drop of glTF models to your feedhttps://developers.facebook.com/docs/sharing/3d-posts/

Over 150K royalty-free glTF modelshttps://sketchfab.com/features/gltf

Dimension generates glTF for delivery of 3D marketing assetshttps://theblog.adobe.com/behind-the-scenes-with-adobe-dimension-engineers-how-we-built-the-3d-publish-feature/

Integrating glTF into ‘HUBS’ Web VR Meeting Space and ‘Spoke’ VR Authoring Toolhttps://www.roadtovr.com/mozillas-hubs-one-click-vr-meeting-space-ive-waiting/

OGC liaison and 3D Tiles OGC Community Standard candidate https://github.com/AnalyticalGraphicsInc/3d-tiles/

Microsoft makes glTF files as pervasively usable as JPGs in Windows 10Open sources entire glTF SDK! https://github.com/Microsoft/glTF-SDK

Open source WebComponent 3D model viewer Import of glTF into AR Core apps via the Google Sceneform Tools plugin

https://github.com/GoogleWebComponents/model-viewer

© The Khronos® Group Inc. 2018 - Page 17This work is licensed under a Creative Commons Attribution 4.0 International License

Blender glTF 2.0 Exporter

glTF IO shipping with Blender 2.8 Beta! https://developer.blender.org/rBAb1f2133fa2849da272e9d8404f371220226ddaf1

© The Khronos® Group Inc. 2018 - Page 18This work is licensed under a Creative Commons Attribution 4.0 International License

Focus on glTF Ecosystem Robustness• Khronos constantly working on improving ecosystem's consistency

- Rendering (reference viewer, reference environment) - Technical low-level issues (validator & asset generator)

• If you are CREATING glTF Files- Ensure generated files are validator clean

- https://github.com/KhronosGroup/glTF-Validator- Help the community understand what your exporter supports

- https://github.com/KhronosGroup/glTF/issues/1271

• If you are LOADING glTF files- Ensure loader can correctly load all sample models (integration tests)

- https://github.com/KhronosGroup/glTF-Sample-Models- Ensure loader can correctly load all asset generator models (unit tests)

- https://github.com/bghgary/glTF-Asset-Generator

Users of glTF can help to keep glTF reliable and consistent!

© The Khronos® Group Inc. 2018 - Page 19This work is licensed under a Creative Commons Attribution 4.0 International License

Rendering quality output

glTF Reference Viewer• Generates accurate Ground Truth renderings of glTF Models

- https://github.com/KhronosGroup/glTF-WebGL-PBR/tree/reference-viewer

• Not all glTF apps and engines need visual consistency- But is critical to key use cases such as online retail

• Headless mode, generates images - Compare against offline path-traced renderings - For regression testing

• Can be embedded in Visual Studio for live model previews

© The Khronos® Group Inc. 2018 - Page 20This work is licensed under a Creative Commons Attribution 4.0 International License

glTF Roadmap and Extensions• glTF manages its roadmap very carefully – complexity is the enemy

- Mission #1: ensure widespread, consistent, reliable usage

- Extension for your own use – create a Vendor Extension- Register your PREFIX by submitting a GitHub Issue

- Multiple applications from multiple vendors need an extension- Create a multi-vendor “EXT” extension

- Broadly applicable across all apps/platforms - propose a “KHR” extension- Need at least two implementations- Discussed and agreed by glTF working group - Covered by Khronos Intellectual Property Framework

- Always have a fallback to core spec- Avoid breaking compatibility with

broader ecosystem- If you choose to not have a fallback

list your extension in extensionsRequired

Stable Core Spec

Domain-specificextensions stay as

extensions

New widely needed functionality ships first as

extensions

Integrate extensions into new core spec only when:1) Widespread need is confirmed by the industry

2) Widespread reliable implementation is enabled (e.g. open source)

© The Khronos® Group Inc. 2018 - Page 21This work is licensed under a Creative Commons Attribution 4.0 International License

Draco glTF Mesh Compression Extension• Library for compressing and decompressing 3D geometric meshes and point clouds

- https://github.com/google/draco

• Google has released Draco encoders and decoders in open source - C++ source code encoder to compress 3D data- C++ and JavaScript decoders for the encoded data- https://github.com/google/draco/tree/gltf_2.0_draco_extension

• Draco glTF extension launched in February 2018- Draco designed and built for compression efficiency and speed - great fit with glTF!- https://github.com/KhronosGroup/glTF/blob/master/extensions/2.0/Khronos/KHR_draco_mesh_compression/README.md

Mesh Compression

Ratios

© The Khronos® Group Inc. 2018 - Page 22This work is licensed under a Creative Commons Attribution 4.0 International License

glTF Roadmap Discussions• Many of these are being discussed on GitHub

- Come and give your views! https://github.com/KhronosGroup/glTF

• Universal Compressed Texture Transmission Format CTTF)- https://github.com/KhronosGroup/glTF-Texture-Transmission-Tools/milestone/1

• PBR 2.0 – advanced materials- Call for key use cases: https://github.com/KhronosGroup/glTF/issues/1442

• Animation 2.0- Including Avatars and Face emoji, with compression

• Metadata• LOD and Streaming• Point Clouds (with compression)

NVIDIA MDL Physically Based Rendering

glTF Roadmap is Driven by Developer Feedback and Requirements

© The Khronos® Group Inc. 2018 - Page 23This work is licensed under a Creative Commons Attribution 4.0 International License

3D Descriptor Database

With Consumer Capture – 3D Will Go Social!

ObjectCapture

Upload, View, Share and Comment

Inspire and Motivate

3D Printing (e.g. Shapeways.com)

Social Loop Photos -> FacebookVideos -> YouTube3D -> glTF!

Manufacturers provide 3D Object Descriptors - much

more information than Amazon-style 2D search

Search

Print

© The Khronos® Group Inc. 2018 - Page 24This work is licensed under a Creative Commons Attribution 4.0 International License

Thank You!• Khronos is creating cutting-edge royalty-free open standards

- For 3D, VR/AR, Compute, Vision and Machine Learning

• Please join Khronos!- We welcome members from Japan and Asia- Influence the direction of industry standards- Get early access to draft specifications- Network with industry-leading companies

• More Information- www.khronos.org- ntrevett@nvidia.com- @neilt3d

Khronos Group Japan Local Member MeetingOpen to current and prospective members!

Thursday, 6 December 20185:00 PM – 8:00 PM JST

Yurakucho Cafe & DiningB1F, Tokyo International ForumChiyoda-ku, Tōkyō-to 100-0005Please Register for Free!