video analysis in autonomous systems: data analytics challenges
DESCRIPTION
Presentation given at "Data Analytics Challenges" workshop in School of Mathematics, University of Leeds.TRANSCRIPT
School of somethingFACULTY OF OTHERSchool of ComputingFACULTY OF ENGINEERING
Video Analysis in Autonomous Systems: Data Analytics Challenges
Krishna Dubba
Institute for Artificial Intelligence and Biological Systems
*
School of ComputingFACULTY OF ENGINEERING
Leeds Activity Analysis Group
Computer Vision (Prof. David Hogg)
Knowledge Representation and Reasoning (Prof. Tony Cohn)
*
School of ComputingFACULTY OF ENGINEERING
Motivation:
“We are drowning in data yet starving for knowledge” ~ John Naisbitt
*
School of ComputingFACULTY OF ENGINEERING
Motivation:● Are computers drowning in (video) data?
○ CCTV cameras○ Personal digital video cameras○ Video content on TV and Internet○ In future: Google glass, autonomous cars, personal
robots
*
School of ComputingFACULTY OF ENGINEERING
TrixiUniversity of Hamburg
LUCIELeeds University Cognitive Intelligent Entity
*
School of ComputingFACULTY OF ENGINEERING
Motivation:● Are computers starving for knowledge?
*
School of ComputingFACULTY OF ENGINEERING
Motivation:● Applications:
○ Security and Surveillance○ Intelligent autonomous systems (robots, cars etc.)○ Content based video retrieval (instead of text tags)○ Automatic script and commentary generation for videos
*
School of ComputingFACULTY OF ENGINEERING
Nature of Data:● Images ● Each pixel in image is a tuple (R,G,B)
*
School of ComputingFACULTY OF ENGINEERING
Nature of Data:● Videos (series of images)
*
School of ComputingFACULTY OF ENGINEERING
Nature of Data:● Videos (series of images)
Third Person View
*
School of ComputingFACULTY OF ENGINEERING
Nature of Data:● Videos (series of images)
Third Person View Ego-Centric View
*
School of ComputingFACULTY OF ENGINEERING
Nature of Data● Sensor data such as laser, depth data etc (Kinect).
*
School of ComputingFACULTY OF ENGINEERING
Nature of Data● Sensor data such as laser, depth data etc (Kinect).
*
School of ComputingFACULTY OF ENGINEERING
Nature of Data:
● Text (annotations, additional information from web)
● Verbal instructions
*
School of ComputingFACULTY OF ENGINEERING
Challenges:● Supervised, unsupervised and semi-supervised learning
*
School of ComputingFACULTY OF ENGINEERING
Challenges:● Supervised, unsupervised and semi-supervised learning● Data comes from multiple sources and mainly aimed at
humans - Multidisciplinary approach
*
School of ComputingFACULTY OF ENGINEERING
Challenges:● Supervised, unsupervised and semi-supervised learning● Data comes from multiple sources and mainly aimed at
humans - Multidisciplinary approach● Real time analysis: GPU processing
○ LUCIE has three kinects attached and needs a separate computer for each kinect.
*
School of ComputingFACULTY OF ENGINEERING
Challenges:● Supervised, unsupervised and semi-supervised learning● Data comes from multiple sources and mainly aimed at
humans - Multidisciplinary approach● Real time analysis: GPU processing
○ LUCIE has three kinects attached and needs a separate computer for each kinect.
● Integrating low-level representation and high level reasoning: Statistical Relational Models like Markov Logic Networks
*
School of ComputingFACULTY OF ENGINEERING
Challenges:● Supervised, unsupervised and semi-supervised learning● Data comes from multiple sources and mainly aimed at
humans - Multidisciplinary approach● Real time analysis: GPU processing
○ LUCIE has three kinects attached and needs a separate computer for each kinect.
● Integrating low-level representation and high level reasoning: Statistical Relational Models like Markov Logic Networks
● Online learning and how learning affects the state of the system.
*
Thank You
School of ComputingFACULTY OF ENGINEERING