User Interfaces

for 3D

K. Gatland and D. JefferisThe World of the Future: Robots.1979, Usborne Hayes.

Outline• Why bother with 3D ?

• Some Basic Tasks

• Input devices

• Output devices

• Properties and pitfalls of 3D

• Scientific Visualization

• Information Visualization in 3D

• Interaction techniques and interface schemes

• Demos

Why bother with 3D ?• Displaying objects or environments that are

naturally 3D (architectural plans, industrial designs, …). Examples:– using VR to shop for a new kitchen

– designing an automobile

– visualizing CT scans or MRI data

http://www.tjhsst.edu/TechLabs/CAD/cad98/rciszek/ahome.htm

• Scientific Visualization– Where the data has a natural 3D spatial structure

•Sometimes, 3D isn’t enough !

Why bother with 3D ?

Two colliding black holes.http://jean-luc.ncsa.uiuc.edu/Exhibits/exhibits.html

Weyl scalar fields from orbiting binary neutron stars.http://jean-luc.ncsa.uiuc.edu/Movies/

sphere eversion

• To harness natural human abilities ?– Pre-conscious processing by the human visual system– Spatial memory

Why bother with 3D ?

The cow jumped over the moon.

The moon is the largest natural satellite of the earth, and is composed of 30 % cheddar, 40 % mozzarella, 25 % star dust, and 5 % Elmer’s glue. Yesterday, at 12:15 pm, the cow owned by Mrs. Farmwell jumped over the moon.

Example:

•To not use 3D seems like a waste of bandwidth !

•But how do we represent abstract data in 3D ?

http://www.angelfire.com/pa2/klb01/spheregallery2.html

Why bother with 3D ?• We can pack more information, and more

complex relationships, into 3D– Information visualization

• Where abstract data is embedded into 3D

• Example below: Anemone (Benjamin Fry)

Using the process of organic information design to visualize the changing structure of a web site,juxtaposed with usage information.http://acg.media.mit.edu/people/fry/anemone/

• Added complexity

• Many more degrees of freedom to handle

• “Standard” input/output devices not designed for 3D

• Interfaces may be difficult to learn

Why not bother with 3D ?

The benefits shouldoutweigh the costs !

Some Basic Tasks

• Specify a point (3 DOF)

• Specify an orientation (3 DOF)

• Specify a path

• Selecting an object or region in space

• Create an object or surface

• Navigation (6 DOF)

• Finding out where something is

Navigation: Camera Control

Zoom vs Dolly (Translation)

From slides by Chris North

Metaphors for Camera Navigation

• World-in-hand

• Eyeball-in-hand (a.k.a. egocentric)

• Walking

• Flying

Some Input Devices

• Plain old 2D mouse– Use picking ray to select objects– Translate a 3D cursor, 1 or 2 dimensions at a time

• 2+1 D pointers– E.g. mouse + thumbwheel; lever that can be

pushed/pulled

• 3D pointers– E.g. trackers, floating mice, …

• Higher DOF devices– Data glove, shape tape, …

Spaceball; Logitech Magellan

•6 DOF, but …

•Rate control rather than position control

http://www.alsos.com/Products/Devices/SpaceBall.html

Rockin’ Mouse

R. Balakrishnan, T. Baudel, G. Kurtenbach, G. Fitzmaurice (1997). The Rockin’ Mouse: Integral 3D manipulation on a plane. CHI’97.

Shape Tape

“Doll’s Head”: Props-based interface for 3D Cutting Plane

• Ken Hinckley

From slides by Chris North

Output devices• Flat screens

• Stereoscopic displays– Red/green glasses, LCD shutters, head-mounted

displays– Autostereoscopic displays

• Re-imaging displays

• Parallax displays (e.g. holographic displays)

• Volumetric displays

For more info on autostereoscopic displays:

http://web.media.mit.edu/~halazar/autostereo/autostereo.html

High Fidelity 3D Output

• Stereoscopic

• Convergence

• Accommodation

Boom Chameleon(G. Fitzmaurice et al.)

•Navigation is easy to learn

•3D view can be shared

•Not stereoscopic

G. Fitzmaurice and fakespacesystems.com

Virtual Reality

Head-mounted display High DOF input device

•“Immersive”, but also cumbersome ?

•Stereoscopic, but no ocular accommodation

Volumetric Display

The Perspecta display. www.actuality-systems.com

•Stereoscopic, and ocular accommodation !

•No occlusion, and no view independent shading

•Interesting property: no perspective projections possible

Volumetric Display

Elizabeth Downing, www.3dtl.com

Properties and Pitfalls of 3D

What is this ?

•What’s behind it ? What’s on the other side ?

•What’s behind me ?

•3D is inherently subjective

Where am I ?

Where am I ?

Where am I ?

Cues (visual, and depth)• Occlusion

– Gives ordinal information

• Transparency• Perspective

– Relative size, foreshortening, converging lines

• Stereopsis• Motion parallax• Contour, shading, specular highlights, reflections• Shadows (e.g. drop shadows)• Ground plane grid, coloured sky• Landmarks, compass arrows

Example use of cues

Vida Dujmovi\'{c}, Pat Morin, David R. WoodPath-Width and Three-Dimensional Straight-Line Grid Drawings of GraphsGD 2002

Example use of cues: card readers

Example use of cues:Shading with surface normals

(images by Michael McGuffin)

Example use of cues

Product Logohttp://www.cri-mw.co.jp/products/product_adx_e.htm

Example use of cues

Plumb Design’s

Visual Thesaurus

http://www.visualthesaurus.com/

Scientific Visualization

From an ad for GRAFTOOL software, made by 3-D Visions, appearing in the January 1992 issue of Scientific American.

From an ad for GRAFTOOL software, made by 3-D Visions, appearing in the January 1992 issue of Scientific American.

From an ad for GRAFTOOL software, made by 3-D Visions, appearing in the January 1992 issue of Scientific American.

Cutting Planes & Isosurfaces

http://www.slicerdicer.com/2astrolg.html

Geological Data

http://www.slicerdicer.com/6geolg.html

Volumetric Data

Janet Haswell“Visualizing Electromagnetic Data”inG. Grinstein and H. Levkowitz (Eds.)“Perceptual Issues in Visualization”pp. 109--1251995Springer

Textures for enhacing cues

L. M. de la Cruz, I. Garcia, V. Godoy, E. Ramos, “Case study: parallel lagrangian visualization applied to natural convective flows”, ACM PVG 2001

Sphere Eversion

http://www.geom.umn.edu/~munzner/ieee94/ieee/node25.html

Sphere Eversion

http://www.geom.umn.edu/graphics/pix/Video_Productions/Outside_In/blue-red-alpha.html

Hyperbolic Space

http://www.geom.umn.edu/~munzner/ieee94/ieee/node25.html

Hierarchical Flow Diagram

PhD thesis of H. Loeffelmannhttp://www.vrvis.at/vis/

Medical Vis

C. Balazs et al.

http://www.vrvis.at/vis/research/npvr/

Medical Vis

C. Balazs et al.

http://www.vrvis.at/vis/research/npvr/

Information Visualization in 3D

If Keanu Reeves does it, it must be cool, no ?

3D message board

•Looks cool, but …

•User spends most of their time navigating

•How to fix ?

3D message board

A static diagram of theProthos Application Framework

http://www.cs.ualberta.ca/~hoover/cmput660/readings/SoftwareArch/section/prothos.htm

http://www.cs.ualberta.ca/~hoover/cmput660/readings/SoftwareArch/image/framework.gif

What problems would arise if this diagram were dynamic/interactive ?

Collapsible Cylindrical Trees(Dachselt and Ebert, IEEE InfoVis 2001)

ThreeDFM

http://www.icewalkers.com/Linux/Software/516470/ThreeDFM.html

File System Navigator (FSN) by SGI

From slides by Chris North http://www.sgi.com/fun/freeware/3d_navigator.html

File System Visualizer (FSV) http://fsv.sourceforge.net/

Information Pyramids

Andrews and Wolte and Pichler

IEEE Visualization 1997, Late Breaking Hot Topics

Cone Trees (G. Robertson et al.)

G. Robertson, J. Mackinlay, and S. Card. Cone trees: Animated 3d visualizations of hierarchical information. CHI '91.Image courtesy of G. Fitzmaurice.

Kodisein’s file browser

http://www.mulle-kybernetik.com/software/kodisein/manual/filebrowser.html

Innolab 3D File Manager(Ferris wheel-like arrangement)

Adam Miezianko, Kristopher Rambish, Karen Fung, Zavnura Pingkanhttp://www.sciencemag.org/cgi/content/full/301/5639/1476http://www.sciencemag.org/feature/data/vis2003/illus_first.htmlhttp://www.sciencemag.org/feature/data/vis2003/images/fileman_large.jpg

“Hierarchical Net”

Hierarchy based 3D Visualization of Large Software StructuresMichael Balzer and Oliver Deussenposter at Vis 2004

PolyPlane (Hong and Murtagh,IEEE InfoVis 2003 poster)

Information Cube (Rekimoto & Green)

Adapted from slides by Chris North

Cube: a 3D visual programming language (Marc Najork)

http://www.research.compaq.com/SRC/personal/najork/cube.html

SAM: An Animated 3D Programming Language (Christian Geiger et al.)

“3D-PP” Visual Programming System (Oshiba and Tanaka)

Force-directed layout of graphs in 3D

http://www.dgp.toronto.edu/~mjmcguff/research/graph3D/

•Pseudo-physical simulation of forces leads to automatic layout

•Nodes are mutually repelled by an electrical force

•Edges are springs

Valence (Ben Fry)•Nodes are words in a text

•Edges connect words that appear consecutively in the text (thus, the text is a path through the graph)

•Words that appear many times are pushed outward

•Pairs of words that appear many times have shorter edges connecting them

•Graph is constructed dynamically as text is read in

Image from Ben Fry’s master’s thesis

Valence (Benjamin Fry)

http://acg.media.mit.edu/people/fry/valence/

Gradus (Matt Grenby)

•Scatter plot of dictionary words

•x, y, z axes correspond to time, familiarity, alphabetic ordering

•Overall form reveals something about underlying data

http://acg.media.mit.edu/people/grenby/gradus/

Examples ofInteraction Techniques,

Interface Schemes

(Very) Indirect ManipulationThrough Traditional Widgets

• Creates a divided attention problem

•User must switch between mouse and keyboard

Thomas Strothotte. “Computational Visualization: Graphics, Abstraction, and Interactivity”, p. 318

Indirect ManipulationThrough 3D Widgets

“Cosmo Player” VRML browser, by SGI

http://hackberry.chem.trinity.edu/IJC/Text/adam.gif

http://www.refractions.net/terrainserver/screenshots/snapshot_vrml.jpghttp://www.mmu.ac.uk/art-des/arc/people/sforestiero/diss/FIG05.JPG

(Almost) Direct Manipulation Through 3D Widgets

“Manipulators” in Maya for translating, rotating, and scaling a cube

•All operations performed with a regular mouse

Traditional WidgetsEmbedded in the 3D scene

http://www.ssec.wisc.edu/~billh/sc95.gif

Smarter Cameras

• Camera always stays upright

• Camera detects surrounding geometry

• Glances (J. Pierce et al.)

• StyleCam (N. Burtnyk et al.)

J. S. Pierce, M. Conway, M. Van Dantzich, G. Robertson.Toolspaces and Glances.1999 Symposium on Interactive 3D Graphics (I3D '99).

Voodoo Dolls (Jeff Pierce)

•“Crushing head” selection

•Other possible schemes: casting fishing line

J. S. Pierce, B. Stearns, R. Pausch.Two Handed Manipulation of Voodoo Dolls in Virtual Environments.1999 Symposium on Interactive 3D Graphics (I3D '99).

Haptic feedback: The Phantom

http://www.sensable.comR. Jagnow and J. Dorsey.Virtual sculpting with haptic displacement maps.Proceedings of Graphics Interface, 2002.

3D desktop (3dna.net)

•Improved harnessing of spatial memory ?

Win3D, by clockwise3d.com

3D OS X

http://www.acm.uiuc.edu/macwarriors/projects/3dosx/screenshots.html

Project Looking Glass(Sun Microsystems)

http://wwws.sun.com/software/looking_glass/

A 3D Window Manager(www.3dwm.org)

www.3dwm.org

www.3dwm.org

www.3dwm.org

Ferris Wheel layout

http://www.lego.com/eng/create/designer/default.asp?x=x&id=4100

Metisse (Olivier Chapuis and Nicolas Roussel)

http://insitu.lri.fr/~chapuis/metisse/screenshots/

Data Mountain (G. Robertson et al.)

G. Robertson et al.Data Mountain: Using spatial memory for document management. UIST ’98.

"Our pre-attentive ability to recognize spatial relationships[...] makes it possible to place pages at a distance(thereby using less screen space) and understand theirspatial relationships without thinking about it."

Task Gallery (G. Robertson et al.)

G. Robertson et al.The Task Gallery: A 3D Window Manager. CHI 2000.

2D versus 3D (Cockburn et al.)

A. Cockburn and B. McKenzie.Evaluating the effectivenessof spatial memory in 2D and 3Dphysical and virtual environments. CHI 2002.

Proprioception and VR

Reference for above pictures: Mine et al., "Moving objects in space: exploiting proprioception in virtual-environment interaction", SIGGRAPH '97. For related work, see also Pierce, Conway, van Dantzich, Robertson (1999), Toolspaces and Glances, I3D’99

CSCW in 3D• Goal: awareness of location & view of other

users in a common 3D space

• Avatars not enough, because– When avatar is distant or facing away, can’t see

what user is looking at– When avatar is out of view, can’t tell anything

about the user

Groupspace• 7 techniques to increase awareness of

location, perspective, and proximity of others:– Nose ray– View cone– Head light– Awareness slider– Rotating participant– WYSIWIS participant– Grand tour

Groupspace

J. Dyck and C. Gutwin. Groupspace: a 3D workspace supporting user awareness. CHI 2002.

Lessons ?• For things that are “naturally” 3D (e.g. cars, scientific

visualization)– 3D is necessary, but still not simple– Good design will make it better

• For information visualization,– 3D can be worse than 2D– Good design is essential

• Good design includes– Use of constraints where possible, to simplify navigation

and manipulation– Use of cues, to ease interpretation of visual information

Demos

Extra Material

http://dform1shiftfunc.net

http://dform1shiftfunc.net

Doom as a tool for system administration(Dennis Chao)

http://www.cs.unm.edu/~dlchao/flake/doom/

Doom as a tool for system administration(Dennis Chao)

http://www.cs.unm.edu/~dlchao/flake/doom/

Doom as a tool for system administration

Advantages:• “The machine load is immediately apparent to the player,

who can see how crowded a room is.”• “A new sysadmin can be given less power by providing

her with a smaller weapon.”• “Drastic action takes work. In a command line interface,

all actions take approximately the same amount of effort. One can ls just as easily as rm -rf * […] In a cyberspace environment […] performing large actions takes time and effort.”

• “Important processes can be instantiated as more powerful monsters. They can then defend themselves against inexperienced sysadmins.”

Doom as a tool for system administration

Disadvantages:

• “Mapping processes to appropriate monsters is difficult. Should large processes be mapped to large monsters? Should the monster type reflect the CPU as well as memory usage? Should processes and their children look alike?”

• “It is difficult to tell if your employees are doing real work or just goofing off when tools and games have the same GUI.”