Download - Agents of Change: HIT Lab Graduate Students Mark Billinghurst HIT Lab., University of Washington
Agents of Change:HIT Lab Graduate Students
Mark Billinghurst
HIT Lab., University of Washington
HITL Consortium 98
Skills for the New Millenium
Creative is more valuable than smart … “if you're in the top 1%, there's only 55 million people smarter than you”...
Change Agents Need: flexibility broad knowledge Learn to see the world from as many
viewpoints as possible Work and play nice with others
HITL Consortium 98
Current Students
PhD H. Abi-Rached, J. Berkley, M. Billinghurst,
W. Chinthammit, H. Duh, D. Gatica-Perez, N. Hedley, R. Jackson, D. Kim, J. King, H. Pryor, S. Ruiz
Masters J. Brandt, M. Brown, G. Dilby, K. Kloeckner,
L. Matheson, S. Tanney
HITL Consortium 98
Interface
Jessica Baldis - Spatial Audio Conferencing Matt Brown - 4D Waccom Mouse Jimena Olveres - Expressive Avatars Susan Tanney - Architecture as Interface
HITL Consortium 98
Spatial Audio Conferencing
Traditional Video Conferencing audio same performance as audio + video different from face to face
Why ? redundancy among cues lack of spatial cues
What happens when spatial cues are added?
HITL Consortium 98
Hypothesis
Spatial audio cues should help with: memory focal assurance perceived comprehension user preference
HITL Consortium 98
Experimental Conditions
Non-spatial Co-located spatial Scaled spatial
HITL Consortium 98
Results
Spatial audio improved all measures: increasing memory, focal assurance, user
preferences. But:
little difference between co-located and scaled conditions
no difference between visual and non-visual condtions
HITL Consortium 98
Results
Audio Condition
ScaledCo-LocatedNon-Spatial
Mea
n R
espo
nse
7
6
5
4
3
2
1
Audio Condition
ScaledCo-LocatedNon-Spatial
Mea
n sp
eake
r id
entif
icat
ion
scor
e.
26
24
22
20
18
16
14
12
10
8
6
4
2
0
Amount of attention
Speaker Identification
HITL Consortium 98
Wacom 4D Mouse
New Input Device 5 buttons and thumbwheel used with tablet, pen graphic design applications
HIT Lab Goals explore new interface metaphors compare performance to traditional mouse
HITL Consortium 98
HITL Consortium 98
HITL Consortium 98
Expressive Avatars
Face-to-Face Conversation People don’t think their expressions Use of nonverbal cues Simultaneous non-verbal and verbal input
Virtual Environment Conversation Avatar representation Non-verbal input explicitly entered. Sequential non-verbal and verbal input
HITL Consortium 98
And they lived happily ever after ...
Avatar expressions are hard: smiling, but not laughing, frowning, crying ...
HITL Consortium 98
Intelligent, Expressive Avatars
Goal: To add intelligent, expressive avatars to text based chat environments.
Proposal: Develop an Intelligent System which infers different
emotions from textual input. Use emotion information to automatically set the
graphical appearance of the avatars.
HITL Consortium 98
How it Works
Text Input
NLP Parsing-key word spotting-phrase length-emoticon spotting
Expert System-emotion hypothesis-context information Animation Engine
Emotion Scores-joy -interest-sadness -surprise-anger -shyness-fear
Virtual Avatar 3D(C, C++, OpenGL)
User Input
Synthesized speech
HITL Consortium 98
Avatar Expressions
Emotional Displays - happy and angry.
HITL Consortium 98
User Testing
Users recognize the emotions: 6 emotions recognized with 95% accuracy.
Users enjoyed the interaction: 75% Enjoyed the intelligent system. 15% Too complicated. 10% They would not used again.
HITL Consortium 98
Architecture As InterfaceVirtual Playground
Loose interpretation of urban, architectural, and theatrical metaphors urban scale with respect to private and
public spaces architectural features suggesting openings
help to define the pedestrian (user) scale proscenium interface from 3D environment
to 2D webpage
Usable architecture Data displayed as inhabitable
structures, patterns, and orders. Spatial relationships such as
proximity, scale, shape, and movement suggest meaning.
Dynamic architectures that inform the user.
HITL Consortium 98
Architecture As InterfaceFlight of the Phoenix
Cooperative (or hybrid) architectures Physical augmenting the virtual Virtual augmenting the physical
Animate Architecturesor...what I learned from CSE 458
What if walls could speak?
How can we use animation in virtual
environments to assist the user? Could the
architecture serve as a live database?
HITL Consortium 98
Hardware
Homer Pryor - VRD Emulator Kyle Kloeckner - Low Vision Studies Sal Ruiz - VRD Head/Eye Tracker
HITL Consortium 98
VRD Color System
ArgonLaser
Delivery Optics
C&D Electronics
Scanners
AO Modulators
Red LaserDiode
HITL Consortium 98
VRD Emulator
HITL Consortium 98
Amblyopia
Percent Difference- Amblyopic Subjects (A5,A8)
-20.000.00
20.0040.0060.00
3.15 1.88 1.22 0.74
Character Size (Angle Subtended)
Per
cen
t (%
)
VRD1,CRTW
VRD1,CRTR
VRD2,CRTW
VRD2,CRTR
HITL Consortium 98
Keratoconus
Keratoconus Testing (S2)
0
1
2
3
4
5
6
7
8
Paper CRT VRD Paper CRT VRD Paper CRT Paper CRT VRD
Vis
ua
l Ac
uit
y
OSOD
Contact Lens W/O Contact Lens Pinhole GlassesContact Lens W/O Contact Pinhole Glasses
HITL Consortium 98
VRD for Head/Eye Tracking
Head Tracking Alternatives VRD as scanning aperture to single detector VRD illuminating reflective markers VRD illuminating active sensors
Eye Tracking Alternatives scan IR into eye along with visible VRD display scan aperture of single linear arrays across
eye for high speed eye position
HITL Consortium 98
Software
Daniel Gatica-Perez - Image Segmentation Habib Abi-Rached - Panoramic Displays Jeff Berkley - FE Force Feedback Mark Billinghurst - Conversational Interfaces
HITL Consortium 98
Video Segmentation Goal
Segment video into meaningful semantic objects Method
Mathematical Morphology Techniques Applications
Video Conferencing Extract objects from video sequences and add them to
other video or virtual environments MPEG4 - object based video compression
HITL Consortium 98
Segmentation Results
Original Video
Segmented Video
HITL Consortium 98
Improving Automobile Visibility
HITL Consortium 98
Blending
HITL Consortium 98
Stereo Acuity / Depth Perception
HITL Consortium 98
Results
Large variance in results 15% people no errors more than 50% had 100% error
Must look at other cues mirrors kept for certain people panoramic displays for others (>50%)
HITL Consortium 98
Percent Of Errors
2.688-3.000
2.408-2.688
2.158-2.408
1.933-2.158
1.732-1.933
1.552-1.732
1.390-1.552
1.246-1.390
1.116-1.246
1.000-1.116
3.5-4.9
4.9-6.8
6.8-9.6
9.6-13.4
13.4-18.7
18.7-26.2
26.2-36.6
36.6-51.1
51.1-71.5
71.5-100.0
0
5
10
15
20
25
30
35
perc
en
tag
e
Zratio
Znear
Percent of errors
HITL Consortium 98
Surgical Tissue Simulation
Goals: Develop new “fast finite element” algorithms for
modeling real-time tissue deformation. Validate the fast FE-based deformation and
force-feedback methods. Create an intuitive software platform for
development of simple surgical simulations.
HITL Consortium 98
Significance
Learning surgery is difficult can’t practice on real tissue !
Surgical simulators are valuable but realistic looking rather than real time
FE modeling can provide realistic deformation and force-feedback
But methods are needed for optimizing FE equations to achieve real-time results.
HITL Consortium 98
Initial Results
FE haptic deformation incorporated into FLIGHT software algorithm optimized to run in real-time on an O2, rather
than 4 processor Onyx Force feedback methods validated in Sinus surgery
simulator
HITL Consortium 98
Conversational Characters
Rea: The Real Estate Agent vision and speech input conversational understanding voice and gesture output
Emphasis on Function not a one to one mapping
between input and interpretation
Figure 5. User Interacting with Rea
HITL Consortium 98
Conclusion
HIT Lab culture breeds great students Talented Diverse Creative Agents of Change
Companies which realize this: Phillips, Disney, Intel, AT&T, Microsoft, JPL … Insert Your Name Here