Agents of Change:HIT Lab Graduate Students

Mark Billinghurst

HIT Lab., University of Washington

HITL Consortium 98

Skills for the New Millenium

Creative is more valuable than smart … “if you're in the top 1%, there's only 55 million people smarter than you”...

Change Agents Need: flexibility broad knowledge Learn to see the world from as many

viewpoints as possible Work and play nice with others

HITL Consortium 98

Current Students

PhD H. Abi-Rached, J. Berkley, M. Billinghurst,

W. Chinthammit, H. Duh, D. Gatica-Perez, N. Hedley, R. Jackson, D. Kim, J. King, H. Pryor, S. Ruiz

Masters J. Brandt, M. Brown, G. Dilby, K. Kloeckner,

L. Matheson, S. Tanney

HITL Consortium 98

Interface

Jessica Baldis - Spatial Audio Conferencing Matt Brown - 4D Waccom Mouse Jimena Olveres - Expressive Avatars Susan Tanney - Architecture as Interface

HITL Consortium 98

Spatial Audio Conferencing

Traditional Video Conferencing audio same performance as audio + video different from face to face

Why ? redundancy among cues lack of spatial cues

What happens when spatial cues are added?

HITL Consortium 98

Hypothesis

Spatial audio cues should help with: memory focal assurance perceived comprehension user preference

HITL Consortium 98

Experimental Conditions

Non-spatial Co-located spatial Scaled spatial

HITL Consortium 98

Results

Spatial audio improved all measures: increasing memory, focal assurance, user

preferences. But:

little difference between co-located and scaled conditions

no difference between visual and non-visual condtions

HITL Consortium 98

Results

Audio Condition

ScaledCo-LocatedNon-Spatial

Mea

n R

espo

nse

7

6

5

4

3

2

1

Audio Condition

ScaledCo-LocatedNon-Spatial

Mea

n sp

eake

r id

entif

icat

ion

scor

e.

26

24

22

20

18

16

14

12

10

8

6

4

2

0

Amount of attention

Speaker Identification

HITL Consortium 98

Wacom 4D Mouse

New Input Device 5 buttons and thumbwheel used with tablet, pen graphic design applications

HIT Lab Goals explore new interface metaphors compare performance to traditional mouse

HITL Consortium 98

HITL Consortium 98

HITL Consortium 98

Expressive Avatars

Face-to-Face Conversation People don’t think their expressions Use of nonverbal cues Simultaneous non-verbal and verbal input

Virtual Environment Conversation Avatar representation Non-verbal input explicitly entered. Sequential non-verbal and verbal input

HITL Consortium 98

And they lived happily ever after ...

Avatar expressions are hard: smiling, but not laughing, frowning, crying ...

HITL Consortium 98

Intelligent, Expressive Avatars

Goal: To add intelligent, expressive avatars to text based chat environments.

Proposal: Develop an Intelligent System which infers different

emotions from textual input. Use emotion information to automatically set the

graphical appearance of the avatars.

HITL Consortium 98

How it Works

Text Input

NLP Parsing-key word spotting-phrase length-emoticon spotting

Expert System-emotion hypothesis-context information Animation Engine

Emotion Scores-joy -interest-sadness -surprise-anger -shyness-fear

Virtual Avatar 3D(C, C++, OpenGL)

User Input

Synthesized speech

HITL Consortium 98

Avatar Expressions

Emotional Displays - happy and angry.

HITL Consortium 98

User Testing

Users recognize the emotions: 6 emotions recognized with 95% accuracy.

Users enjoyed the interaction: 75% Enjoyed the intelligent system. 15% Too complicated. 10% They would not used again.

HITL Consortium 98

Architecture As InterfaceVirtual Playground

Loose interpretation of urban, architectural, and theatrical metaphors urban scale with respect to private and

public spaces architectural features suggesting openings

help to define the pedestrian (user) scale proscenium interface from 3D environment

to 2D webpage

Usable architecture Data displayed as inhabitable

structures, patterns, and orders. Spatial relationships such as

proximity, scale, shape, and movement suggest meaning.

Dynamic architectures that inform the user.

HITL Consortium 98

Architecture As InterfaceFlight of the Phoenix

Cooperative (or hybrid) architectures Physical augmenting the virtual Virtual augmenting the physical

Animate Architecturesor...what I learned from CSE 458

What if walls could speak?

How can we use animation in virtual

environments to assist the user? Could the

architecture serve as a live database?

HITL Consortium 98

Hardware

Homer Pryor - VRD Emulator Kyle Kloeckner - Low Vision Studies Sal Ruiz - VRD Head/Eye Tracker

HITL Consortium 98

VRD Color System

ArgonLaser

Delivery Optics

C&D Electronics

Scanners

AO Modulators

Red LaserDiode

HITL Consortium 98

VRD Emulator

HITL Consortium 98

Amblyopia

Percent Difference- Amblyopic Subjects (A5,A8)

-20.000.00

20.0040.0060.00

3.15 1.88 1.22 0.74

Character Size (Angle Subtended)

Per

cen

t (%

)

VRD1,CRTW

VRD1,CRTR

VRD2,CRTW

VRD2,CRTR

HITL Consortium 98

Keratoconus

Keratoconus Testing (S2)

0

1

2

3

4

5

6

7

8

Paper CRT VRD Paper CRT VRD Paper CRT Paper CRT VRD

Vis

ua

l Ac

uit

y

OSOD

Contact Lens W/O Contact Lens Pinhole GlassesContact Lens W/O Contact Pinhole Glasses

HITL Consortium 98

VRD for Head/Eye Tracking

Head Tracking Alternatives VRD as scanning aperture to single detector VRD illuminating reflective markers VRD illuminating active sensors

Eye Tracking Alternatives scan IR into eye along with visible VRD display scan aperture of single linear arrays across

eye for high speed eye position

HITL Consortium 98

Software

Daniel Gatica-Perez - Image Segmentation Habib Abi-Rached - Panoramic Displays Jeff Berkley - FE Force Feedback Mark Billinghurst - Conversational Interfaces

HITL Consortium 98

Video Segmentation Goal

Segment video into meaningful semantic objects Method

Mathematical Morphology Techniques Applications

Video Conferencing Extract objects from video sequences and add them to

other video or virtual environments MPEG4 - object based video compression

HITL Consortium 98

Segmentation Results

Original Video

Segmented Video

HITL Consortium 98

Improving Automobile Visibility

HITL Consortium 98

Blending

HITL Consortium 98

Stereo Acuity / Depth Perception

HITL Consortium 98

Results

Large variance in results 15% people no errors more than 50% had 100% error

Must look at other cues mirrors kept for certain people panoramic displays for others (>50%)

HITL Consortium 98

Percent Of Errors

2.688-3.000

2.408-2.688

2.158-2.408

1.933-2.158

1.732-1.933

1.552-1.732

1.390-1.552

1.246-1.390

1.116-1.246

1.000-1.116

3.5-4.9

4.9-6.8

6.8-9.6

9.6-13.4

13.4-18.7

18.7-26.2

26.2-36.6

36.6-51.1

51.1-71.5

71.5-100.0

0

5

10

15

20

25

30

35

perc

en

tag

e

Zratio

Znear

Percent of errors

HITL Consortium 98

Surgical Tissue Simulation

Goals: Develop new “fast finite element” algorithms for

modeling real-time tissue deformation. Validate the fast FE-based deformation and

force-feedback methods. Create an intuitive software platform for

development of simple surgical simulations.

HITL Consortium 98

Significance

Learning surgery is difficult can’t practice on real tissue !

Surgical simulators are valuable but realistic looking rather than real time

FE modeling can provide realistic deformation and force-feedback

But methods are needed for optimizing FE equations to achieve real-time results.

HITL Consortium 98

Initial Results

FE haptic deformation incorporated into FLIGHT software algorithm optimized to run in real-time on an O2, rather

than 4 processor Onyx Force feedback methods validated in Sinus surgery

simulator

HITL Consortium 98

Conversational Characters

Rea: The Real Estate Agent vision and speech input conversational understanding voice and gesture output

Emphasis on Function not a one to one mapping

between input and interpretation

Figure 5. User Interacting with Rea

HITL Consortium 98

Conclusion

HIT Lab culture breeds great students Talented Diverse Creative Agents of Change

Companies which realize this: Phillips, Disney, Intel, AT&T, Microsoft, JPL … Insert Your Name Here