augmented reality collaborative environment: calibration and interactive scene editing

37
iMAGIS-GRAVIR / IMAG Augmented Reality Collaborative Environment: Calibration and Interactive Scene Editing Raphaël Grasset Xavier Decoret Jean-Dominique Gascuel iMAGIS-GRAVIR/IMAG iMAGIS is a joint project between CNRS,INRIA,INPG and UJF

Upload: ugo

Post on 02-Feb-2016

30 views

Category:

Documents


0 download

DESCRIPTION

Augmented Reality Collaborative Environment: Calibration and Interactive Scene Editing. Raphaël Grasset Xavier Decoret Jean-Dominique Gascuel iMAGIS-GRAVIR/IMAG. iMAGIS is a joint project between CNRS,INRIA,INPG and UJF. Motivations. Focus on : - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Augmented Reality  Collaborative Environment:  Calibration and Interactive Scene Editing

iMAGIS-GRAVIR / IMAG

Augmented Reality Collaborative Environment:

Calibration and Interactive Scene Editing

Raphaël GrassetXavier DecoretJean-Dominique Gascuel

iMAGIS-GRAVIR/IMAG

iMAGIS is a joint project between CNRS,INRIA,INPG and UJF

Page 2: Augmented Reality  Collaborative Environment:  Calibration and Interactive Scene Editing

iMAGIS-GRAVIR / IMAG

Motivations

Focus on: – tracking and registration of virtual and real objects.– commercial age: off the shelf and personal hardware.

Purpose: propose a collaborative environment for multi-user in augmented reality.

Page 3: Augmented Reality  Collaborative Environment:  Calibration and Interactive Scene Editing

iMAGIS-GRAVIR / IMAG

Related Works

Some Augmented Reality Environments

•GRASP[1995]

•Studierstube[1998]

•EMMIE[1999]

Page 4: Augmented Reality  Collaborative Environment:  Calibration and Interactive Scene Editing

iMAGIS-GRAVIR / IMAG

Overview•Motivation and Related Works

•Setup: Small Group and Computing Environment

•Calibration Procedure– HMD Calibration– Pen Calibration– World Calibration

•Dynamic Objects– Mixing Real and Virtual objects– Moving Objects

•Conclusion

Page 5: Augmented Reality  Collaborative Environment:  Calibration and Interactive Scene Editing

iMAGIS-GRAVIR / IMAG

Setup: Small GroupCentralized Configuration:

– user around a table: face to face collaboration.– personal and shared area.

Shared area

Personal area(virtual menu, private data)

Page 6: Augmented Reality  Collaborative Environment:  Calibration and Interactive Scene Editing

iMAGIS-GRAVIR / IMAG

Setup: Computing Environment

•Visualization: optical see-through HMDdirect view, private information, low weight.

•Interaction: stylus

easy use.

•Tracking: magnetic tracker

fast, no line of sight.

Page 7: Augmented Reality  Collaborative Environment:  Calibration and Interactive Scene Editing

iMAGIS-GRAVIR / IMAG

Overview•Motivation and Related Works

•Setup: Small Group and Computing Environment

•Calibration Procedure– HMD Calibration– Pen Calibration– World Calibration

•Dynamic Objects– Mixing Real and Virtual objects– Moving Objects

•Conclusion

Page 8: Augmented Reality  Collaborative Environment:  Calibration and Interactive Scene Editing

iMAGIS-GRAVIR / IMAG

Calibration Procedure

World CS

Image CS

Transmitter CS

Receptor CS

Emitter CS

Eye CSHMD CS

Tip CS

Purpose: obtain the relations between different coordinate systems (CS).

HMD CS = middle between the 2 optic centers of the HMD.

Style: fast and accurate interactive procedure.

Page 9: Augmented Reality  Collaborative Environment:  Calibration and Interactive Scene Editing

iMAGIS-GRAVIR / IMAG

Calibration Procedure

World CS

Image CS

Transmitter CS

Receptor CS

Emitter CS

Eye CSHMD CS

Tip CSHMD CS = middle between the 2 optic centers of the HMD.

Style: fast and accurate interactive procedure.

HMD Calibration

Purpose: obtain the relations between different coordinate systems (CS).

Page 10: Augmented Reality  Collaborative Environment:  Calibration and Interactive Scene Editing

iMAGIS-GRAVIR / IMAG

Calibration Procedure

World CS

Image CS

Transmitter CS

Receptor CS

Emitter CS

Eye CSHMD CS

Tip CSHMD CS = middle between the 2 optic centers of the HMD.

Style: fast and accurate interactive procedure.

Pen Calibration

Purpose: obtain the relations between different coordinate systems (CS).

Page 11: Augmented Reality  Collaborative Environment:  Calibration and Interactive Scene Editing

iMAGIS-GRAVIR / IMAG

Calibration Procedure

World CS

Image CS

Transmitter CS

Receptor CS

Emitter CS

Eye CSHMD CS

Tip CSHMD CS = middle between the 2 optic centers of the HMD.

Style: fast and accurate interactive procedure.World Calibration

Purpose: obtain the relations between different coordinate systems (CS).

Page 12: Augmented Reality  Collaborative Environment:  Calibration and Interactive Scene Editing

iMAGIS-GRAVIR / IMAG

Related Works

Some Calibration Methods:

•Azuma[1994] & Oishi[1996]: lot of hardware.

•Tuceryan[1995]: video see-through method

Page 13: Augmented Reality  Collaborative Environment:  Calibration and Interactive Scene Editing

iMAGIS-GRAVIR / IMAG

Related Works

•Fuhrmann[1999] (Studierstube): calibration with a tracked pen.

•Genc[2000] (GRASP): vision approach.

Page 14: Augmented Reality  Collaborative Environment:  Calibration and Interactive Scene Editing

iMAGIS-GRAVIR / IMAG

HMD CalibrationPurpose: transformation between the Receptor CS and the Image CS.Style: reduce user effort.

Constraints: personnal hardware => removable tracker on the HMD.

Depends on the position of the tracker in the HMD (HMD CS Receptor CS), Morphology ..etc..

Page 15: Augmented Reality  Collaborative Environment:  Calibration and Interactive Scene Editing

iMAGIS-GRAVIR / IMAG

HMD Calibration: Procedure

Receptor CS

HMD CS

Eye CS

Emitter CS

Misalignment

Method: freeze the view and align a virtual object with real object.

Virtual Emitter

1.Freeze the view

Page 16: Augmented Reality  Collaborative Environment:  Calibration and Interactive Scene Editing

iMAGIS-GRAVIR / IMAG

HMD Calibration: Procedure

Receptor CS

HMD CS

Eye CS

Emitter CS

Virtual Emitter

Freeze View

2. Move head

Method: freeze the view and align a virtual object with real object.

Page 17: Augmented Reality  Collaborative Environment:  Calibration and Interactive Scene Editing

iMAGIS-GRAVIR / IMAG

HMD Calibration: Procedure

Receptor CS

HMD CS

Eye CS

Emitter CS

Virtual Emitter

Freeze View

3. Unfreeze the view

Method: freeze the view and align a virtual object with real object.

Page 18: Augmented Reality  Collaborative Environment:  Calibration and Interactive Scene Editing

iMAGIS-GRAVIR / IMAG

HMD Calibration: Procedure

Receptor CS

HMD CS

Eye CS

Emitter CS

Virtual Emitter

Incremental realignment.

4. Compute transformation

Method: freeze the view and align a virtual object with real object.

Page 19: Augmented Reality  Collaborative Environment:  Calibration and Interactive Scene Editing

iMAGIS-GRAVIR / IMAG

Pen Calibration: Principles and Procedures

Purpose: transformation between the Transmitter CS and the Tip CS.

Method: orient the stylus in n representative directions.

Emitter CS

Fixed tip

Compute by SVD (nx3 equations, 6 unknowns).

Page 20: Augmented Reality  Collaborative Environment:  Calibration and Interactive Scene Editing

iMAGIS-GRAVIR / IMAG

Pen Calibration: Results

SVD on 100x3 equations SVD on 4x3 equations

Tracker positions

Reconstruction of the tip

We propose to keep just 4 measurements.

Page 21: Augmented Reality  Collaborative Environment:  Calibration and Interactive Scene Editing

iMAGIS-GRAVIR / IMAG

World CalibrationPurpose: transformation between the Emitter CS and the World CS.

Method: selection of 4 corners of shared area.

P1

P2P3

P4

Emitter CSWorld CS

Page 22: Augmented Reality  Collaborative Environment:  Calibration and Interactive Scene Editing

iMAGIS-GRAVIR / IMAG

Calibration Results

Error largely acceptable for most practical applications.

2 cm

Different views of the source square with a virtual figure added: no disturbing drift.

Page 23: Augmented Reality  Collaborative Environment:  Calibration and Interactive Scene Editing

iMAGIS-GRAVIR / IMAG

Overview•Motivation and Related Works

•Setup: Small Group and Computing Environment

•Calibration Procedure– HMD Calibration– Pen Calibration– World Calibration

•Dynamic Objects– Mixing Real and Virtual objects– Moving Objects

•Conclusion

Page 24: Augmented Reality  Collaborative Environment:  Calibration and Interactive Scene Editing

iMAGIS-GRAVIR / IMAG

Dynamics objects: Mixing Real and Virtual Environment

Purpose: – Describe real objects to the system.– Tangible User Interface.– Poor man haptic feedback.

Requirements:– Occlusion compatibility (« black render »).– Respect of laws of physics (e.g gravity). – Manipulation awareness.

Our approach: interactive system easy use based on visual feedback.

• Digitalized objects.• Generic Geometric models.• TUI objects.

Page 25: Augmented Reality  Collaborative Environment:  Calibration and Interactive Scene Editing

iMAGIS-GRAVIR / IMAG

Related WorksSome interactions with real objects

•Whitaker[1995]: image-based and pointer-based object calibration.

•Mixed Reality[1999]: features on objects.

•Build-IT[1997]

Page 26: Augmented Reality  Collaborative Environment:  Calibration and Interactive Scene Editing

iMAGIS-GRAVIR / IMAG

Mixing with Already Digitalized Models (1/2)

Purpose: obtain transformation between the object CS and the world CS (known geometry + appearance).

Constraints: laying on top of something else (3DOF).Method: superpose virtual representation with real representation of the model.

Real object Virtual objectObject CS

Page 27: Augmented Reality  Collaborative Environment:  Calibration and Interactive Scene Editing

iMAGIS-GRAVIR / IMAG

Mixing with New Simple ModelsPurpose: obtain the geometry and the position of generic objects (cube, cylinder, cone, ..etc..)

Our approach: Description of simple shapes with few points.

Example: add a rectangle.

8 DOF

Page 28: Augmented Reality  Collaborative Environment:  Calibration and Interactive Scene Editing

iMAGIS-GRAVIR / IMAG

Mixing with New Simple Models

P1

5 DOF

Purpose: obtain the geometry and the position of generic objects (cube, cylinder, cone, ..etc..)

Our approach: Description of simple shape with few points.

Example: add a rectangle.

Page 29: Augmented Reality  Collaborative Environment:  Calibration and Interactive Scene Editing

iMAGIS-GRAVIR / IMAG

Mixing with New Simple Models

P1

5 DOF

Purpose: obtain the geometry and the position of generic objects (cube, cylinder, cone, ..etc..)

Our approach: Description of simple shape with few points.

Example: add a rectangle.

Page 30: Augmented Reality  Collaborative Environment:  Calibration and Interactive Scene Editing

iMAGIS-GRAVIR / IMAG

Mixing with New Simple Models

P1

2 DOF

P2

Purpose: obtain the geometry and the position of generic objects (cube, cylinder, cone, ..etc..)

Our approach: Description of simple shape with few points.

Example: add a rectangle.

Page 31: Augmented Reality  Collaborative Environment:  Calibration and Interactive Scene Editing

iMAGIS-GRAVIR / IMAG

Mixing with New Simple Models

P1

0 DOF

P2

Purpose: obtain the geometry and the position of generic objects (cube, cylinder, cone, ..etc..)

Our approach: Description of simple shape with few points.

Example: add a rectangle.

Page 32: Augmented Reality  Collaborative Environment:  Calibration and Interactive Scene Editing

iMAGIS-GRAVIR / IMAG

Mixing with New Simple Models: Results

Page 33: Augmented Reality  Collaborative Environment:  Calibration and Interactive Scene Editing

iMAGIS-GRAVIR / IMAG

Tangible User Interface:Moving Objects

Purpose: use real object for manipulation of virtual models.Constraints: one tracker + several real objects to manipulate.

Our approach: TEMPORARILY stick the magnetic tracker to previously added objects.

Real object

Page 34: Augmented Reality  Collaborative Environment:  Calibration and Interactive Scene Editing

iMAGIS-GRAVIR / IMAG

Video

Page 35: Augmented Reality  Collaborative Environment:  Calibration and Interactive Scene Editing

iMAGIS-GRAVIR / IMAG

ConclusionOur contributions• Multi-user environment setup.• Fast user calibration method with sufficient accuracy

for a large range of non-critical applications.• Dynamic placement and digitalization of objects.• Intuitive metaphor to manipulate objects.

Future works• Develop interaction techniques.• Reduce latency.

Page 36: Augmented Reality  Collaborative Environment:  Calibration and Interactive Scene Editing

iMAGIS-GRAVIR / IMAG

Page 37: Augmented Reality  Collaborative Environment:  Calibration and Interactive Scene Editing

iMAGIS-GRAVIR / IMAG

Mixing with already Digitalized Models (2/2)

Real object