active capture and folk computing
DESCRIPTION
Active Capture and Folk Computing. Ana Ramírez and Marc Davis ICME 2004 – Taipei, Taiwan 29 June 2004. UC Berkeley - Garage Cinema Research - Group for User Interface Research. Smart Multimedia Acquisition Systems. First two papers – automatic camera calibration Image Audio - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Active Capture and Folk Computing](https://reader036.vdocument.in/reader036/viewer/2022070405/56813cf9550346895da69e27/html5/thumbnails/1.jpg)
Active Capture and Folk Active Capture and Folk ComputingComputing
Ana Ramírez and Marc Davis
ICME 2004 – Taipei, Taiwan29 June 2004
UC Berkeley - Garage Cinema Research - Group for User Interface Research
![Page 2: Active Capture and Folk Computing](https://reader036.vdocument.in/reader036/viewer/2022070405/56813cf9550346895da69e27/html5/thumbnails/2.jpg)
6/29/2004 2
Smart Multimedia Acquisition Smart Multimedia Acquisition SystemsSystems• First two papers – automatic camera
calibration Image Audio
• Third paper – understand structure of what is being captured to edit in real time
• Active capture - smart cameras that interactively guide and capture human action
![Page 3: Active Capture and Folk Computing](https://reader036.vdocument.in/reader036/viewer/2022070405/56813cf9550346895da69e27/html5/thumbnails/3.jpg)
6/29/2004 3
OutlineOutline
• Sample applications
• Active Capture
• Designing Active Capture algorithms
• Future work
![Page 4: Active Capture and Folk Computing](https://reader036.vdocument.in/reader036/viewer/2022070405/56813cf9550346895da69e27/html5/thumbnails/4.jpg)
6/29/2004 4
SampleSample ApplicationsApplicationsAutomatic Movie TrailersAutomatic Movie Trailers
Play Video
• Video of capture process
![Page 5: Active Capture and Folk Computing](https://reader036.vdocument.in/reader036/viewer/2022070405/56813cf9550346895da69e27/html5/thumbnails/5.jpg)
6/29/2004 5
SampleSample ApplicationsApplicationsAutomatic Movie TrailersAutomatic Movie Trailers
Play Video
• Video of automatically created movie trailer
![Page 6: Active Capture and Folk Computing](https://reader036.vdocument.in/reader036/viewer/2022070405/56813cf9550346895da69e27/html5/thumbnails/6.jpg)
6/29/2004 6
SampleSample ApplicationsApplicationsSports InstructionSports Instruction
![Page 7: Active Capture and Folk Computing](https://reader036.vdocument.in/reader036/viewer/2022070405/56813cf9550346895da69e27/html5/thumbnails/7.jpg)
6/29/2004 7
SampleSample ApplicationsApplicationsTelemedicineTelemedicine
Rural Town
Large City
leishmaniasis
![Page 8: Active Capture and Folk Computing](https://reader036.vdocument.in/reader036/viewer/2022070405/56813cf9550346895da69e27/html5/thumbnails/8.jpg)
6/29/2004 8
SampleSample ApplicationsApplicationsAutomated Health ScreeningAutomated Health Screening
Rural Town
Large City
leishmaniasis
![Page 9: Active Capture and Folk Computing](https://reader036.vdocument.in/reader036/viewer/2022070405/56813cf9550346895da69e27/html5/thumbnails/9.jpg)
6/29/2004 9
Active CaptureActive Capture
Capture Interaction
Processing
Direction/Cinematography
Human-Computer Interaction
Computer Vision/
Audition
ActiveCapture
![Page 10: Active Capture and Folk Computing](https://reader036.vdocument.in/reader036/viewer/2022070405/56813cf9550346895da69e27/html5/thumbnails/10.jpg)
6/29/2004 10
Active CaptureActive Capture
Capture Interaction
Processing
Direction/Cinematography
Human-Computer Interaction
Computer Vision/
Audition
ActiveCapture
![Page 11: Active Capture and Folk Computing](https://reader036.vdocument.in/reader036/viewer/2022070405/56813cf9550346895da69e27/html5/thumbnails/11.jpg)
6/29/2004 11
Active CaptureActive Capture
Capture Interaction
Processing
Direction/Cinematography
Human-Computer Interaction
Computer Vision/
Audition
ActiveCapture
![Page 12: Active Capture and Folk Computing](https://reader036.vdocument.in/reader036/viewer/2022070405/56813cf9550346895da69e27/html5/thumbnails/12.jpg)
6/29/2004 12
Active CaptureActive Capture
Capture Interaction
Processing
Direction/Cinematography
Human-Computer Interaction
Computer Vision/
Audition
ActiveCapture
![Page 13: Active Capture and Folk Computing](https://reader036.vdocument.in/reader036/viewer/2022070405/56813cf9550346895da69e27/html5/thumbnails/13.jpg)
6/29/2004 13
Active CaptureActive Capture
Capture Interaction
Processing
Direction/Cinematography
Human-Computer Interaction
Computer Vision/
Audition
ActiveCapture
![Page 14: Active Capture and Folk Computing](https://reader036.vdocument.in/reader036/viewer/2022070405/56813cf9550346895da69e27/html5/thumbnails/14.jpg)
6/29/2004 14
Active CaptureActive Capture• Traditionally, signal processing
algorithms avoid interacting with the user
• Signal processing + interaction => more sophisticated recognizers
• How to design hybrid algorithms that involve capture, interaction, and processing
![Page 15: Active Capture and Folk Computing](https://reader036.vdocument.in/reader036/viewer/2022070405/56813cf9550346895da69e27/html5/thumbnails/15.jpg)
6/29/2004 15
Components of Active Capture Components of Active Capture AlgorithmsAlgorithms
• Simple computer vision and audition recognizers / sensors Motion Eyes Sound
• Desired action in terms of recognizers
• Interaction script
![Page 16: Active Capture and Folk Computing](https://reader036.vdocument.in/reader036/viewer/2022070405/56813cf9550346895da69e27/html5/thumbnails/16.jpg)
6/29/2004 16
Design ProcessDesign Process
• Input: Desired action = head turn Recognizers = motion, eyes
MotionEyes
time
![Page 17: Active Capture and Folk Computing](https://reader036.vdocument.in/reader036/viewer/2022070405/56813cf9550346895da69e27/html5/thumbnails/17.jpg)
6/29/2004 17
Design ProcessDesign Process
• Input: Desired action = head turn Recognizers = motion, eyes
• Step 1: Express desired action in terms of
recognizers
MotionEyes
time
No Motion
No Eyes
Motion
Eyes
No Motion
![Page 18: Active Capture and Folk Computing](https://reader036.vdocument.in/reader036/viewer/2022070405/56813cf9550346895da69e27/html5/thumbnails/18.jpg)
6/29/2004 18
Design ProcessDesign Process
• Input: Desired action = head turn Recognizers = motion, eyes
• Step 1: Express desired action in terms of
recognizers
• Step 2: Design interaction script
![Page 19: Active Capture and Folk Computing](https://reader036.vdocument.in/reader036/viewer/2022070405/56813cf9550346895da69e27/html5/thumbnails/19.jpg)
6/29/2004 19
Design Process – Step IIDesign Process – Step II
time
no motion
head turning
no motion
"Please face left..." "Turn to face the camera..."
1.5 - 4 sec
> 1.5 sec< 1 sec
< 4.5 sec< 3.5 sec
> 0.25 sec
"Great!"
> 0.9 sec
capture
Motion
Eyes
0.25-1.25s
0.25-2.75s
no eyes
eyes
“GoAhead”
"Please stand still..."motion
eyes "Please look away..."
0.25-2.75s"Please look at end of turn..."
0.25-2.75s"Didn’t look at camera..."
0.25-2.75s"Please turn slower..."
"Didn’t move..."0.25-2.75s
![Page 20: Active Capture and Folk Computing](https://reader036.vdocument.in/reader036/viewer/2022070405/56813cf9550346895da69e27/html5/thumbnails/20.jpg)
6/29/2004 20
Design Process – Step IIDesign Process – Step II
time
no motion
head turning
no motion
"Please face left..." "Turn to face the camera..."
1.5 - 4 sec
> 1.5 sec< 1 sec
< 4.5 sec< 3.5 sec
> 0.25 sec
"Great!"
> 0.9 sec
capture
Motion
Eyes
0.25-1.25s
0.25-2.75s
no eyes
eyes
“GoAhead”
"Please stand still..."motion
eyes "Please look away..."
0.25-2.75s"Please look at end of turn..."
0.25-2.75s"Didn’t look at camera..."
0.25-2.75s"Please turn slower..."
"Didn’t move..."0.25-2.75s
![Page 21: Active Capture and Folk Computing](https://reader036.vdocument.in/reader036/viewer/2022070405/56813cf9550346895da69e27/html5/thumbnails/21.jpg)
6/29/2004 21
Design Process – Step IIDesign Process – Step II
time
no motion
head turning
no motion
"Please face left..." "Turn to face the camera..."
1.5 - 4 sec
> 1.5 sec< 1 sec
< 4.5 sec< 3.5 sec
> 0.25 sec
"Great!"
> 0.9 sec
capture
Motion
Eyes
0.25-1.25s
0.25-2.75s
no eyes
eyes
“GoAhead”
"Please stand still..."motion
eyes "Please look away..."
0.25-2.75s"Please look at end of turn..."
0.25-2.75s"Didn’t look at camera..."
0.25-2.75s"Please turn slower..."
"Didn’t move..."0.25-2.75s
Play Video
![Page 22: Active Capture and Folk Computing](https://reader036.vdocument.in/reader036/viewer/2022070405/56813cf9550346895da69e27/html5/thumbnails/22.jpg)
6/29/2004 22
Design Process – Step IIDesign Process – Step II
time
no motion
head turning
no motion
"Please face left..." "Turn to face the camera..."
1.5 - 4 sec
> 1.5 sec< 1 sec
< 4.5 sec< 3.5 sec
> 0.25 sec
"Great!"
> 0.9 sec
capture
Motion
Eyes
0.25-1.25s
0.25-2.75s
no eyes
eyes
“GoAhead”
"Please stand still..."motion
eyes "Please look away..."
0.25-2.75s"Please look at end of turn..."
0.25-2.75s"Didn’t look at camera..."
0.25-2.75s"Please turn slower..."
"Didn’t move..."0.25-2.75s
![Page 23: Active Capture and Folk Computing](https://reader036.vdocument.in/reader036/viewer/2022070405/56813cf9550346895da69e27/html5/thumbnails/23.jpg)
6/29/2004 23
Design Process – Step IIDesign Process – Step II
time
no motion
head turning
no motion
"Please face left..." "Turn to face the camera..."
1.5 - 4 sec
> 1.5 sec< 1 sec
< 4.5 sec< 3.5 sec
> 0.25 sec
"Great!"
> 0.9 sec
capture
Motion
Eyes
0.25-1.25s
0.25-2.75s
no eyes
eyes
“GoAhead”
"Please stand still..."motion
eyes "Please look away..."
0.25-2.75s"Please look at end of turn..."
0.25-2.75s"Didn’t look at camera..."
0.25-2.75s"Please turn slower..."
"Didn’t move..."0.25-2.75s
Play Video
![Page 24: Active Capture and Folk Computing](https://reader036.vdocument.in/reader036/viewer/2022070405/56813cf9550346895da69e27/html5/thumbnails/24.jpg)
6/29/2004 24
Design ChallengesDesign Challenges
Step I - Description of action Approximate timing Strict and non strict ordering
Step II – Interaction script What to do if something goes wrong –
mediation
![Page 25: Active Capture and Folk Computing](https://reader036.vdocument.in/reader036/viewer/2022070405/56813cf9550346895da69e27/html5/thumbnails/25.jpg)
Step I – Action DescriptionStep I – Action Description
![Page 26: Active Capture and Folk Computing](https://reader036.vdocument.in/reader036/viewer/2022070405/56813cf9550346895da69e27/html5/thumbnails/26.jpg)
6/29/2004 26
• Observations• Commands• Capture• Time constraints• Strict ordering• Non-strict ordering
Step I - Action DescriptionStep I - Action DescriptionVisual LanguageVisual Language
![Page 27: Active Capture and Folk Computing](https://reader036.vdocument.in/reader036/viewer/2022070405/56813cf9550346895da69e27/html5/thumbnails/27.jpg)
6/29/2004 27
Step I - Action DescriptionStep I - Action DescriptionVisual LanguageVisual Language• Observations• Commands• Capture• Time constraints• Strict ordering• Non-strict ordering
![Page 28: Active Capture and Folk Computing](https://reader036.vdocument.in/reader036/viewer/2022070405/56813cf9550346895da69e27/html5/thumbnails/28.jpg)
6/29/2004 28
Step I - Action DescriptionStep I - Action DescriptionVisual LanguageVisual Language• Observations• Commands• Capture• Time constraints• Strict ordering• Non-strict ordering
![Page 29: Active Capture and Folk Computing](https://reader036.vdocument.in/reader036/viewer/2022070405/56813cf9550346895da69e27/html5/thumbnails/29.jpg)
6/29/2004 29
Step I - Action DescriptionStep I - Action DescriptionVisual LanguageVisual Language• Observations• Commands• Capture• Time constraints• Strict ordering• Non-strict ordering
![Page 30: Active Capture and Folk Computing](https://reader036.vdocument.in/reader036/viewer/2022070405/56813cf9550346895da69e27/html5/thumbnails/30.jpg)
6/29/2004 30
Step I - Action DescriptionStep I - Action DescriptionVisual LanguageVisual Language• Observations• Commands• Capture• Time constraints• Strict ordering• Non-strict ordering
![Page 31: Active Capture and Folk Computing](https://reader036.vdocument.in/reader036/viewer/2022070405/56813cf9550346895da69e27/html5/thumbnails/31.jpg)
6/29/2004 31
Step I - Action DescriptionStep I - Action DescriptionVisual LanguageVisual Language
time
waving
noise
"Say your name and wave"
> 4.5 sec
capture
Waving
Noise
<1 sec <1 sec
• Observations• Commands• Capture• Time constraints• Strict ordering• Non-strict ordering
![Page 32: Active Capture and Folk Computing](https://reader036.vdocument.in/reader036/viewer/2022070405/56813cf9550346895da69e27/html5/thumbnails/32.jpg)
6/29/2004 32
Step I - Action DescriptionStep I - Action DescriptionVisual LanguageVisual Language
time
waving
noise
"Say your name and wave"
> 4.5 sec
capture
Waving
Noise
<1 sec <1 sec
• Observations• Commands• Capture• Time constraints• Strict ordering• Non-strict ordering
![Page 33: Active Capture and Folk Computing](https://reader036.vdocument.in/reader036/viewer/2022070405/56813cf9550346895da69e27/html5/thumbnails/33.jpg)
Step II – Interaction ScriptStep II – Interaction Script
![Page 34: Active Capture and Folk Computing](https://reader036.vdocument.in/reader036/viewer/2022070405/56813cf9550346895da69e27/html5/thumbnails/34.jpg)
6/29/2004 34
Step II – Interaction ScriptStep II – Interaction ScriptContextual InquiriesContextual Inquiries• Golf instructor• Aikido instructor• 911 emergency phone operator• Triage nurse• Children’s portrait photographer• Film and theatre directors
[Jeffrey Heer, Nathaniel S. Good, Ana Ramirez, Marc Davis, and Jennifer Mankoff. “Presiding Over Accidents: System Direction of Human Action.” In: Proceedings of the Conference on Human Factors in Computing Systems (CHI 2004) in Vienna, Austria. ACM Press, 463-470, 2004. ]
![Page 35: Active Capture and Folk Computing](https://reader036.vdocument.in/reader036/viewer/2022070405/56813cf9550346895da69e27/html5/thumbnails/35.jpg)
6/29/2004 35
Step II – Interaction ScriptStep II – Interaction ScriptDirection and Feedback StrategiesDirection and Feedback Strategies
• External aids
Play Video
![Page 36: Active Capture and Folk Computing](https://reader036.vdocument.in/reader036/viewer/2022070405/56813cf9550346895da69e27/html5/thumbnails/36.jpg)
6/29/2004 36
Step II – Interaction ScriptStep II – Interaction ScriptDirection and Feedback StrategiesDirection and Feedback Strategies
• Decomposition and “Show”
Play Video
![Page 37: Active Capture and Folk Computing](https://reader036.vdocument.in/reader036/viewer/2022070405/56813cf9550346895da69e27/html5/thumbnails/37.jpg)
6/29/2004 37
Step II – Interaction ScriptStep II – Interaction ScriptDirection and Feedback StrategiesDirection and Feedback Strategies
• Method shift from “Show” to “Tell”
Play Video
![Page 38: Active Capture and Folk Computing](https://reader036.vdocument.in/reader036/viewer/2022070405/56813cf9550346895da69e27/html5/thumbnails/38.jpg)
6/29/2004 38
Step II – Interaction ScriptStep II – Interaction ScriptDirection and Feedback StrategiesDirection and Feedback Strategies
![Page 39: Active Capture and Folk Computing](https://reader036.vdocument.in/reader036/viewer/2022070405/56813cf9550346895da69e27/html5/thumbnails/39.jpg)
6/29/2004 39
SummarySummary• Active Capture – smart cameras that
interactively guide and capture human action
• Sample applications Automated health screening Automated movie clips Sports trainer
• Design Challenges Description of action Interaction script
![Page 40: Active Capture and Folk Computing](https://reader036.vdocument.in/reader036/viewer/2022070405/56813cf9550346895da69e27/html5/thumbnails/40.jpg)
6/29/2004 40
Future WorkFuture Work
• Support design and implementation of Active Capture applications
• Evaluate the relative contribution of signal analysis and user interaction in these hybrid algorithms
![Page 41: Active Capture and Folk Computing](https://reader036.vdocument.in/reader036/viewer/2022070405/56813cf9550346895da69e27/html5/thumbnails/41.jpg)
6/29/2004 41
QuestionsQuestions
Ana Ramí[email protected]
www.cs.berkeley.edu/~anar
Garage Cinema Researchhttp://garage.sims.berkeley.edu
Group for User Interface Researchhttp://guir.berkeley.edu