acrv : robotic vision presentation in lisbon at ist
TRANSCRIPT
Juxi Leitner arc centre of excellence for robotic vision
queensland university of technology<[email protected]> http://Juxi.net
robotic vision an australian vision for robotics
ACRV @ UC Berkeley
http://roboticvision.org/
tinyurl.com/QUTRobotics
ICRA 2018 Brisbane, Australia
http://roboticvision.org/
tinyurl.com/QUTRobotics
roboticvision.org
http://roboticvision.org/
http://roboticvision.org/
http://roboticvision.org/
10 96%eyedesigns
animalshave eyes
Nature Reviews | Neuroscience
Protostomes
Bilateria
Ecdysozoa
Lophotrochozoa
~580 Mya
~550 Mya
~530 Mya
~500 Mya
~430 Mya
Deuterostomes
Chordates*
Craniates*
Vertebrates*
Gnathostomes*
Arthropods
Annelids
Molluscs
Hemichordates
Echinoderms
Cephalochordates*
Tunicates*
Myxiniformes*
Petromyzoniformes*
Last fossil jawless fish
Stages of interest in vertebrate eye evolution Cambrian
Mya 600 550 500 450 400 0
2 1 3 4 5 6
Lampreys
Jawed vertebrates
Hagfish
Lancelets
Sea squirts
Ocellus
Eye patch
ProtostomeAn animal belonging to the protostome super-phylum, which is characterized by its members’ embryonic development, in which the first opening (the blastopore) becomes the mouth (protostome is Greek for ‘first mouth’). All protostomes are invertebrates.
DeuterostomeAn animal belonging to the deuterostome super-phylum of the animal kingdom, which is characterized by its members’ embryonic development, in which the first opening (the blastopore) becomes the anus (deuterostome is Greek for ‘second mouth’). In addition to the chordate phylum (which includeds vertebrates), the other two main phyla are the echinoderm phylum and the hemichordate phylum.
ChordateAn animal belonging to the chordate phylum, which comprises vertebrates, tunicates and cephalochordates. These animals are characterized by the presence of a notochord, a dorsal-nerve cord and pharyngeal slits or pouches.
AgnathanA jawless fish within the chordate phylum (agnatha is Greek for ‘no jaw’). The two extant groups are hagfish and lampreys.
GnathostomeThe jawed vertebrates (gnathostome is Greek for ‘jaw mouth’), comprising fish and tetrapods (including birds and mammals).
craniates, hagfish have the most basal body-plan. They possess neither jaws nor vertebrae and are therefore usually regarded not as vertebrates but rather as a sister group. The vertebrates comprise an early jaw-less (agnathan) division, of which the only living examples are lampreys, and a later jawed division, the gnathostomes, which includes fish and tetrapods.
Controversy has long surrounded the interrelation-ship between hagfish, lampreys and jawed vertebrates. BOX 1 summarizes current views, and in FIG. 1 we show hagfish diverging either before the divergence of lam-preys or else after lampreys separated from the line that would become the jawed vertebrates.
Not only has extensive gene duplication occurred throughout the evolution of animals22, but in addition it is widely accepted that two rounds of whole-genome duplication occurred early in vertebrate evolu-tion23–29; most likely, one duplication occurred before the agnathans split from the vertebrate line and one occurred after (FIG. 1; for reviews, see REFS 30–32). It is also clear that the vertebrate organizer, which deter-mines the body plan of developing embryos, arose in early chordates33–35. These genetic developments are likely to have been of crucial importance in early vertebrate evolution, but they are beyond the scope of this Review.
Figure 1 | The origin of vertebrates. The evolution of jawed vertebrates is illustrated against an approximate time-scale of millions of years ago (Mya). The taxa considered in this Review are indicated with an asterisk and are accompanied by schematics and diagrams of the ‘eye’ region. The earliest chordates, represented by extant cephalochordates and tunicates, are thought to have appeared around 550 Mya. Jawless craniates (agnathans) were present in the early Cambrian, by 525 Mya, and a time of 530 Mya has been indicated for their presumed first appearance. As elaborated on in BOX 1, there is considerable controversy as to whether myxiniformes (solely represented by extant hagfish) diverged before or after the separation of lampreys from jawed vertebrates (shown as dashed black and grey lines). Numerous lines of jawless fish evolved between 500 and 430 Mya ago, although none have survived to the present day. The first jawed vertebrate arose around 430 Mya, and this line is represented today by cartilagenous fish, bony fish and tetrapods. Six ‘stages of interest’ in vertebrate eye evolution correspond to the time intervals between the divergence of important surviving taxa. This diagram does not include the evolutionary changes that have occurred in the last 400 million years. The presented timeline is based primarily on evidence from the fossil record; see REFS 2,13,15,17,18,144,160–163. The schematics are modified, with permission, from REF. 11 (1996) Oxford University Press (lancelet, sea squirt, hagfish and lamprey) and REF. 164 (2004) Academic Press (jawed vertebrate). The eye images are reproduced, with permission, from the following references: lancelet, REF. 165 BIODIDAC (1996) University of California Museum of Paleontology; sea squirt, REF. 63 (2006) Blackwell Publishing; hagfish, REF. 166 (2006) Australian Museum. Lamprey and jawed vertebrate eye images are courtesy of G. Westhoff and S. P. Collin).
R E V I E W S
NATURE REVIEWS | NEUROSCIENCE VOLUME 8 | DECEMBER 2007 | 961
VISIONROBOTIC
thanks to G. Metta and IIT for this picture
Robust Vision
Seman/c Representa/ons
Algorithms & Architecture
Visual Learning
Vision and Ac/on
Dalle Molle Institute for AI (IDSIA)
Work
Juxi Leitner
PhD Informatics / Intelligent Systems
MSc Space Robotics & Automation
BSc Information & Software Engineering
Intelligent (Space) Robots European Space Agency (ESA)
Erasmus Intelligent Systems
Work (Humanoid) Robot VisionInstituto Superior Técnico (IST)
Mobility Intelligent Space Systems Laboratory
About Me
Current Robotic Vision and Actions
h:p://Juxi.net
Queensland University of Technology (QUT)
coordinationeye-hand
move to see
see to move
Vision and Ac2on
http://Juxi.net/projects/VisionAndAction/
http://roboticvision.org/
http://roboticvision.org/
http://Juxi.net/special-issue/ijrr-deep-learning-2016/
IJRR Special Issue
R:SS WS(what) scep/cs?
deeper understanding
tool (how to use it?)
RV vs CV
data driven models
understanding limita/ons of deep nets, reinforcement learning & transfer of knowledge
deep learning visual control
Perception Module Control ModuleConv1 Conv2 Conv3 FC_c2 FC_c3FC_c1
Q-v
alue
s
7×7
conv
+ R
eLU
strid
e 2
4×4
conv
+ R
eLU
strid
e 2
3×3
conv
+ R
eLU
strid
e 1
64 lters 64 lters 64 lters
fully
con
n.
300
units
9 un
its
84×84
400
unitsfully
con
n.
fully
con
n. +
ReL
U
fully
con
n. +
ReL
U
I BN
5 un
its
θ
Bottleneck
Or
Occlusion Occlusion Occlusion
[Zhang et al, RA:L (submitted)]
manipulation for improved perception
http://roboticvision.org/
propertiesdetectingvisually[Dansereau, Singh, Leitner, ICRA2016]
http://Juxi.net/projects/squishy/
http://roboticvision.org/
http://Juxi.net/projects/AmazonPickingChallenge/
http://roboticvision.org/
http://Juxi.net/projects/AmazonPickingChallenge/
systemdesign
http://roboticvision.org/
http://Juxi.net/projects/AmazonPickingChallenge/
systemdesign
systemdesign
ObjectDetection
ObjectLocalisation
ObjectIdentification
Shelf Localisation
perception
Kinect2 (once)
RealSense (every attempt)
KinectFusion
2D regions(CNN)
point cloud
segmentation
grasp point detection
http://Juxi.net/projects/AmazonPickingChallenge/
http://roboticvision.org/
score
http://roboticvision.org/
ARC Centre of Excellence for Robotic Vision roboticvision.org
fruit picking
https://www.youtube.com/watch?v=8rq4iSTsg68
ARC Centre of Excellence for Robotic Vision roboticvision.org
limita/ons of current robo/c systems reproducible research on TASKS not datasets
picking benchmarkhttp://Juxi.net/dataset/acrv-picking-benchmark/
http://Juxi.net/dataset/acrv-picking-benchmark/arxiv number!
