fcv taxo perona
TRANSCRIPT
Hilbert problems and challenges
Pietro PeronaCalifornia Institute of Technology
NSF Frontiers in Vision WorkshopCambridge - 21-23 Aug. 2011
Friday, August 26, 2011
Scene understanding
Friday, August 26, 2011
!X
!xO
!x =1
Z!X
Scene understanding
Friday, August 26, 2011
Vision in control loop
Friday, August 26, 2011
Other sensors...
Friday, August 26, 2011
sceneunderstanding vision in the loop
ease of experimentation
goal representation decision/action
active perception
other sensors
Friday, August 26, 2011
Visual recognition
Friday, August 26, 2011
Recognizing..
Objects
Actions
Materials
Scenes
Friday, August 26, 2011
Recognizing..
Objects
Actions
Materials
Scenes
Friday, August 26, 2011
Recognizing..
Objects
Actions
Materials
Scenes
Friday, August 26, 2011
Recognizing..
Objects
Actions
Materials
Scenes
Friday, August 26, 2011
Recognizing..
Objects
Actions
Materials
Scenes
Friday, August 26, 2011
Recognizing..
Objects
Actions
Materials
Scenes
Friday, August 26, 2011
Recognizing..
Objects
Actions
Materials
Scenes
Friday, August 26, 2011
Objects
Actions
Materials
Scenes
Recognizing..
Friday, August 26, 2011
Objects
Actions
Materials
Scenes
Recognizing..
Friday, August 26, 2011
Geometry and materials
Friday, August 26, 2011
How many?
Friday, August 26, 2011
Weaklysupervised
learning
Transferlearning
Friday, August 26, 2011
Incrementallearning
...Friday, August 26, 2011
Subordinate categorization
Friday, August 26, 2011
Organizing visual knowledge
Friday, August 26, 2011
Intentions, causes, consequences, ...
Friday, August 26, 2011
Behavior, events
Friday, August 26, 2011
Friday, August 26, 2011
Friday, August 26, 2011
images, trajectories
pose, movemes, actions, activities, objects, scenes
plans, goals, behavior, relationships ...
SENSO
RYPSY
CH
OLO
GY
Perception
World
interaction, cooperation, competition
Friday, August 26, 2011
group-level goals and plans
individual goals and plans
motor programs
sensor-based control
IMAGING,TRACKING
RECOGNITION
THEORY OF PSYCHOLOGY
SOCIAL NETWORK
PREFRONTAL CORTEX
MOTOR CORTEX
SPINAL CORD
MO
TOR
PLA
NN
ING
Action
INDIVIDUAL
THEORY OF SOCIOLOGY
images, trajectories
pose, movemes, actions, activities, objects, scenes
plans, goals, behavior, relationships ...
SENSO
RYPSY
CH
OLO
GY
Perception
World
interaction, cooperation, competition
Friday, August 26, 2011
Behavior
• Hiearchical representation
• Interactions
• Beyond description: intentions, plans, consequences,
Friday, August 26, 2011
Sharing visual representations with
humans
Friday, August 26, 2011
Friday, August 26, 2011
Friday, August 26, 2011
Grand challenges
Friday, August 26, 2011
Drosophila behavior
[Dankert et al., Nature Methods, April 2009]
Friday, August 26, 2011
Drosophila behavior
[Dankert et al., Nature Methods, April 2009]
Friday, August 26, 2011
VISIPEDIA
Images, segmentsannotations, links,GUIs, diagnostics
Users
AnnotatorsImage databases
Visionscientists
Experts
Automata
Friday, August 26, 2011
<<show movie of traffic in India>>
Autonomous driving
Friday, August 26, 2011
Grand challenges
• Recognition -> Visipedia
• Behavior -> Fly behavior (mouse, ...human)
• Scene understanding -> ???
• Vision for action -> Autonomous driving (how about manipulation?)
• Sustainable economy -> ???
Friday, August 26, 2011
Summary
• Scene understanding vs vision-for-action
• Recognition: just started, much to be done
• Behavior: ditto
• Sharing visual knowledge with humans??
• Grand challenges: many, fun and worthwhile
Friday, August 26, 2011