Rev Lebaredian - Vice President, Simulation Technolgy

SYNTHETIC DATA / AI

2

NEURAL NETWORKS NEED DATA

Deep Learning is amazing!

Require huge amounts of quality data

Data needs labeling

For some problems, data + labeling is available

Good data doesn’t exists for most problems

And Labels!

Image-Net (http://www.image-net.org/) example

3

DOMINOS, ANYONE?Isaac at SIGGRAPH 2017

4

NO EXISTING DATASET

5

Labeled Data

1min per object

7

SYNTHETIC DATA

Real data is expensive, sometimes dangerous

Synthetic labels are automatic and accurate

Useful for validation, in addition to training

8

AV SYNTHETIC DATASETSVirtual KITTI

Adrien Gaidon, Qiao Wang, Yohann Cabon, Eleonora Vig: Virtual Worlds as Proxy for Multi-Object Tracking Analysis IEEE CVPR 2016

9

DOMAIN RANDOMIZATIONExplore the gap using random cars, textures, camera, distractors, etc

10

DOMAIN RANDOMIZATIONExample Scenes

11

STRUCTURED DOMAIN RANDOMIZATIONPutting it all together

12

SIMULATETRAIN MODELSCOLLECT DATA DRIVE

NVIDIA DRIVEEND-TO-END PLATFORM

Lanes Lights

Path

Signs

PedestriansCars

Lanes Lights

Path

Signs

PedestriansCars

13

14

15

AlphaZero OpenAI Five

OpenAI, 2018Deepmind, 2018

REINFORCEMENT LEARNING SUCCESSES

16

APPLICATIONS

Locomotion/Animation

Reinforcement Learning

Liang, Makoviychuk, Handa etc, 2018NVIDIA

17

APPLICATIONS

Sim2Real Robotics

Robotics

Chebotar, Handa, Makoviychuk, etc, 2018NVIDIA

18

EXAMPLESLocomotion

19

EXAMPLESLocomotion + Physics

20

EXAMPLESLocomotion + Physics

21

ISAAC GYMToolkit for Parallel AI Learning Experiments

22

PHYSICSPhysX

23

PHYSICSFleX

• Multi-physics

• Rigid and FEM soft bodies

• Cloth, ropes

• Liquids

• Two-way coupling and force propagation between different phases

24

CABLE ROBOT

25

ROBOTS

LEARN& PLAN

SEE, HEAR,TOUCH

ACT

26

27

28

BARTENDING ROBOTTrained in Isaac Gym

29

ISAAC SIM

30

31

32

33

Our Reality Virtual Reality

Reality

Human

ArtificialIn

telli

gen

ce