3d virtual training systems architecture€¦ · •training systems using real world information...

3D Virtual Training Systems Architecture

January 21-24, 2018

ISO/IEC JTC 1/SC 24/WG 9 & Web3D Meetings

Seoul, Korea

Myeong Won Lee (U. of Suwon)

Virtual Training Systems

• Definition

• Training systems using real world information and 3D virtual environments

• One typical area in systems integration for visualization

• Basic technologies for virtual training systems

• VR/AR/MR

• IoT sensors

• Systems integration

January 21-24, 2019 ISO/IEC JTC 1/SC 24/WG 9 & Web3D Meetings 2

Manufac-turing

Health forMedicine

ground/air/

maritime navigationg

Militarytraining

Sports

Disaster/Safety

Enter-tainment

Application areas of VR AR education

Virtual training system

Haptic

UI/UX Robotics

Motionperception

VR AR

Immersivedevice

Network

Virtual training simulator

Training content

Virtual simulation platform

Experience + Knowledge content

Simulator S/W

Simulator controlManagement tool

Operations managementEvaluation

sba.seoul.kr

Virtual training systems

January 21-24, 2019 3KATS 2015

ISO/IEC JTC 1/SC 24/WG 9 & Web3D Meetings

Systems integration standardization

Virtual education and training systems integration

machinearchi-tecture

road facility interiormanufac-

turingmedicine health

sports harbor aircraft military scienceengi-

neeringelectro-

nicsspace

Real and virtual environment and sensor devices simulation

VR AR

Training system 1

Training system 2

Training system 3

Training system 4

3D Simulation

Application areas of virtual education and training systems

Virtual Education and Training Systems Technologies and Application Areas

• VR/AR/MR integration into training and education systems• Information modeling and exchange• Visualization and simulation• Sensor representation • Real world representation • Graphical user interaction

• Areas related to VR/AR/MR information modeling and rendering• Science and engineering education• Medical education• Training for manipulating industrial equipment or devices

• Ex: flight, car, ship, heavy vehicle, heavy equipment, army, transportation

• Manuals for manipulating electronic and manufacturing products • Ex: camera, video, 3D printer, 3D scanner, home appliance

• Education materials for making products• Ex: clothing, furniture, interior design, food, manually produced products


VR AR Based Education (1)


sites.google.com linkedin.com

clicktoeducation.com

learncube.com webschool.in

Education content and materials Learning and teaching

VR AR Based Training Systems (1)


Driving simulation(Road Traffic Authority, 2013)

CycleOps virtual training(CycleOps, 2012)

Boeing CRVS(The Boeing Company)

Military training(STI, USA)

VR AR Based Training Systems (2)


Excavator simulator(KETI)

Harbor crane training simulator(Total Soft Bank)

Virtual medical education(Virtual Medical Coaching)

Virtual military training

IoT Sensors for Virtual Training Systems (Examples)


• Automotive sensors• Light, temperature, position, ultrasonic, camera, etc.

• Bicycle sensors• Speed, position, light, cadence, etc.

• Flight sensors• Time of flight camera, displacement, inclination, ultrasonic,

rotary, etc.

• Military sensors• Accelerometer, inclination, camera, night vision, etc.

• Medical sensors• Pressure, flow, oxygen, radiation, position, temperature, etc.

• Other VR training area sensors

Standard Technology for Virtual Education and Training Systems


•Virtual environment representation for education and training• VR/AR scene representation• Education and training information representation

•Virtual simulation interface with virtual environments• A virtual scene control interface with sensor

devices

•Virtual simulation with real world environments and sensors• A sensor control interface with sensor devices for

training

• Education and training information database description and manipulation

Virtual Education and Training Systems with IoT Sensors

• Representation of virtual education and training environments• Includes sensor representation

• Interaction with real and virtual education and training environments• Includes interaction with sensors

• Simulation of virtual education and training environments• Includes sensor simulation

• Interfaces for managing and controlling virtual education and training environments• Includes interfaces for managing and controlling sensors


Standards and Standardization for Virtual Education and Training Systems in SC24

• Representation of virtual education and training environments• 3D virtual worlds

• Interaction with real and virtual training environments• IoT sensor and device representation and interactions with 3D

scenes

• Simulation of virtual training environments• 3D visual training, learning, and teaching simulation

• Interfaces for managing and controlling virtual training environments• User interfaces for training, learning, and teaching information

control from/to real and virtual worlds


Education/learning/teaching/training

technology

Architecture for Virtual Training Systems

August 20-23, 2018 ISO/IEC JAG Meeting 13

ICT visua-liza-tion

VR AR MR technology

VR AR MR information

modeling

VR AR MR information

exchange

VR AR MR information interaction

Systems Integration

ICT visu-

alinfoproc

Education technology

Education information

modeling

Education information

exchange

Education information interaction

Virtual education and training systems

ICT information modeling

ICT information exchange

ICT information interaction and interface

Real world device, facility, and information

ICT fundamental technologies(computer system, operating system,

programming language, database, software engineering, network, algorithm,

graphics, multimedia, artificial intelligence, etc.)

Sensor 3D Scene Graph (ISO/IEC CD 18038)

MR Object

Material Geometry

Interfaces with virtual

worlds

Interfaces with real worlds

Physical properties

Shape

An MR Scene

Physical Sensor

3D Object

Material

Geometry

Interfaces with virtual

worlds

Shape

GNSS bounding box info, length and units

GNSS origin

Orientation(pitch/yaw/

roll)

Sensor type

Orientation (pitch/yaw/

roll)

14

MR Object

Interfaces with real worlds

Physical properties

An MR Scene

Physical Sensor

3D Object

GNSS bounding box info, length and units

GNSS origin

Orientation(pitch/yaw/

roll)

Orientation (pitch/yaw/r

oll)

Integrate training simulation with the scene graph

System Framework for Virtual Training Systems

MAR data / education and training data / event streams

Parser

VR AR object

managerEvent controller

Visual object

Sensor object

MAR scene graph manager

3D data Event data

MAR scene / Event graph

External application (to connect

sensor)VR AR scene info

IoT sensor interface

VR AR scene access

interface


Education and training

info

Simulation data

Education simulation manager

3D simulator

Simulation object

Sensor 3D Scene Graph (ISO/IEC CD 18038)

System Framework for Virtual Training Systems

MAR data / education and training data / event streams

Parser

VR AR object

managerEvent controller

Visual object

Sensor object

MAR scene graph manager

3D data Event data

MAR scene / Event graph

External application (to connect

sensor)VR AR scene info

IoT sensor interface

VR AR scene access

interface


Education and training

info

Simulation data

Education simulation manager

3D simulator

Simulation object

New Work Items for Virtual Training Systems

• Information modeling and architecture for virtual training systems• 3D scene graph generation and management

• 3D object management

• Training simulation management in virtual environments

• Sensor event interface

• System framework for virtual training systems


JTC 1 VR AR for Education Ad Hoc Group

• Systems integration methodology for virtual education and training based on JTC1 standards

• JTC1 standards and standardization for virtual training and education systems• SC 24, SC 29, and SC 36 standards and work items

• Other standards would be included

• Guidelines for developing virtual education and training systems using JTC1 standards

• Differences between SC24 and JTC1• SC24: Virtual training systems integration for visualization

• JTC1: Systems integration using all related JTC1 standards for virtual training systems


Conclusions

• Work items for virtual training systems

• Information modeling for virtual education and training systems

• Architecture for virtual education and training systems

• Cooperation with JTC1 VR AR for education ad hoc group


3d virtual training systems architecture€¦ · •training systems using real world information...

Documents