robot vision with cnns: a practical example
DESCRIPTION
Barcelona, 19/2/03. Robot Vision with CNNs: a Practical Example. M. Balsi Dep. of Electronic Engineering “La Sapienza” Univ. of Rome, Italy. P. Vitullo P. Campolucci G. Apicella L. Pompeo D. Bellachioma S. Graziani. X. Vilasís–Cardona S. Luengo J. Solsona R. Funosas. A. Maraschini - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Robot Vision with CNNs: a Practical Example](https://reader035.vdocument.in/reader035/viewer/2022062315/5681597e550346895dc6bcff/html5/thumbnails/1.jpg)
Robot Vision with CNNs:a Practical Example
P. VitulloP. Campolucci
G. ApicellaL. Pompeo
D. BellachiomaS. Graziani
M. BalsiDep. of Electronic Engineering
“La Sapienza” Univ. of Rome, Italy
X. Vilasís–CardonaS. LuengoJ. SolsonaR. Funosas
A. MaraschiniA. Aznar
V. GiovenaleP. Giangrossi
Barcelona, 19/2/03
![Page 2: Robot Vision with CNNs: a Practical Example](https://reader035.vdocument.in/reader035/viewer/2022062315/5681597e550346895dc6bcff/html5/thumbnails/2.jpg)
Framework of this work
• completely autonomous robot• simple (cheap) hardware• vision-based guidance
– short term: line following– longer term: navigation in a real environment
![Page 3: Robot Vision with CNNs: a Practical Example](https://reader035.vdocument.in/reader035/viewer/2022062315/5681597e550346895dc6bcff/html5/thumbnails/3.jpg)
Architecture
• Cellular Neural Networks to handle all the image processing
• Fuzzy-rule-based navigation
![Page 4: Robot Vision with CNNs: a Practical Example](https://reader035.vdocument.in/reader035/viewer/2022062315/5681597e550346895dc6bcff/html5/thumbnails/4.jpg)
Cellular Neural Networks
• Fully parallel analog vision chips• Capable of real-time nonlinear image
processing and feature detection
• Algorithmically programmable to implement complex operations
• On-board image acquisition (focal-plane processing)
![Page 5: Robot Vision with CNNs: a Practical Example](https://reader035.vdocument.in/reader035/viewer/2022062315/5681597e550346895dc6bcff/html5/thumbnails/5.jpg)
Cellular Neural Networks
• Recurrent Neural (?) Network• Locally connected VLSI-friendly• Space-invariant synapses (cloning
templates)– small number of parameters: explicit design
• Continuous variables – analog computing (discrete-time model for digital)
![Page 6: Robot Vision with CNNs: a Practical Example](https://reader035.vdocument.in/reader035/viewer/2022062315/5681597e550346895dc6bcff/html5/thumbnails/6.jpg)
TopologyLocally connected VLSISpace-invariant synapses
![Page 7: Robot Vision with CNNs: a Practical Example](https://reader035.vdocument.in/reader035/viewer/2022062315/5681597e550346895dc6bcff/html5/thumbnails/7.jpg)
Discrete–time model
• Binary state variable• Analog or binary input depending
on implementation
IuB
nxAsignnx
ijNklklljki
ijNklklljkiij
;
;1
![Page 8: Robot Vision with CNNs: a Practical Example](https://reader035.vdocument.in/reader035/viewer/2022062315/5681597e550346895dc6bcff/html5/thumbnails/8.jpg)
Application• Input ports: analog arrays u, x(0)• Output port: binary array x()• “Analog instruction”: {A,B,I} (cloning
template)• Feature detection (nonlinear image
filtering)
IuB
nxAsignnx
ijNklklljki
ijNklklljkiij
;
;1
![Page 9: Robot Vision with CNNs: a Practical Example](https://reader035.vdocument.in/reader035/viewer/2022062315/5681597e550346895dc6bcff/html5/thumbnails/9.jpg)
CNN “Universal” Machine
• Local memory• Global control (broadcasting cloning
templates and memory transfer commands)
• “Analogic” computing: stored-program analog/logic algorithms
![Page 10: Robot Vision with CNNs: a Practical Example](https://reader035.vdocument.in/reader035/viewer/2022062315/5681597e550346895dc6bcff/html5/thumbnails/10.jpg)
Task: line following
• The robot is to follow a maze of straight lines crossing at approximately right angles
• Functions required by vision module:
Acquiring image, cleaning, thinning linesMeasuring orientation/displacement of lines
![Page 11: Robot Vision with CNNs: a Practical Example](https://reader035.vdocument.in/reader035/viewer/2022062315/5681597e550346895dc6bcff/html5/thumbnails/11.jpg)
Image processing algorithm
• Image acquisition
• Binarization
• Line thinning
![Page 12: Robot Vision with CNNs: a Practical Example](https://reader035.vdocument.in/reader035/viewer/2022062315/5681597e550346895dc6bcff/html5/thumbnails/12.jpg)
Image processing algorithm (ctd.)
• Directional line filtering
• Projection
![Page 13: Robot Vision with CNNs: a Practical Example](https://reader035.vdocument.in/reader035/viewer/2022062315/5681597e550346895dc6bcff/html5/thumbnails/13.jpg)
Fuzzy control
![Page 14: Robot Vision with CNNs: a Practical Example](https://reader035.vdocument.in/reader035/viewer/2022062315/5681597e550346895dc6bcff/html5/thumbnails/14.jpg)
Simulation
y (m) z vs. x (m)
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
![Page 15: Robot Vision with CNNs: a Practical Example](https://reader035.vdocument.in/reader035/viewer/2022062315/5681597e550346895dc6bcff/html5/thumbnails/15.jpg)
el cochecito(Barcelona)
control (386)
CNN emul. (DSP)
![Page 16: Robot Vision with CNNs: a Practical Example](https://reader035.vdocument.in/reader035/viewer/2022062315/5681597e550346895dc6bcff/html5/thumbnails/16.jpg)
Visibilia (Rome)
PAL B/WCAMERA
FPGA-based CNN emulatorCeloxica RC-100 board
Xilinx Spartan II 200Kgates
microcontroller
Jackrabbit BL1810
PIC 16F84
SERVOMOTOR
(steering)
LCD
PS/2 mouse port
Rabbit2000microcontroller
Parallel port E
Parallel port ASerial port D
STEPPERMOTOR
(advancing)
STEPPER MOTOR
CONTROLLER
![Page 17: Robot Vision with CNNs: a Practical Example](https://reader035.vdocument.in/reader035/viewer/2022062315/5681597e550346895dc6bcff/html5/thumbnails/17.jpg)
![Page 18: Robot Vision with CNNs: a Practical Example](https://reader035.vdocument.in/reader035/viewer/2022062315/5681597e550346895dc6bcff/html5/thumbnails/18.jpg)
Celoxica RC-100
VGA
![Page 19: Robot Vision with CNNs: a Practical Example](https://reader035.vdocument.in/reader035/viewer/2022062315/5681597e550346895dc6bcff/html5/thumbnails/19.jpg)
Jackrabbit BL1810
![Page 20: Robot Vision with CNNs: a Practical Example](https://reader035.vdocument.in/reader035/viewer/2022062315/5681597e550346895dc6bcff/html5/thumbnails/20.jpg)
drivingstart
vert
hor
follow vert
horY
Y
N
N
horY
N
normal driving
crossing
timer:=0
timer>10s N
Y
store left avail.
turn left if avail.else right
diag (L/R)
Y
follow diagY
N
![Page 21: Robot Vision with CNNs: a Practical Example](https://reader035.vdocument.in/reader035/viewer/2022062315/5681597e550346895dc6bcff/html5/thumbnails/21.jpg)
Continuation of the work
more realistic tasks:• obstacle avoidance• navigation in a real-life environment
![Page 22: Robot Vision with CNNs: a Practical Example](https://reader035.vdocument.in/reader035/viewer/2022062315/5681597e550346895dc6bcff/html5/thumbnails/22.jpg)
Obstacle avoidance• using other sensors together with
vision, e.g. ultrasound• monocular range evaluation• local path-finding strategies
![Page 23: Robot Vision with CNNs: a Practical Example](https://reader035.vdocument.in/reader035/viewer/2022062315/5681597e550346895dc6bcff/html5/thumbnails/23.jpg)
Hybrid (topological/metric) navigation
![Page 24: Robot Vision with CNNs: a Practical Example](https://reader035.vdocument.in/reader035/viewer/2022062315/5681597e550346895dc6bcff/html5/thumbnails/24.jpg)
door recognition
![Page 25: Robot Vision with CNNs: a Practical Example](https://reader035.vdocument.in/reader035/viewer/2022062315/5681597e550346895dc6bcff/html5/thumbnails/25.jpg)
Robot Vision with CNNs:a Practical Example
M. BalsiDep. of Electronic Engineering
“La Sapienza” Univ. of Rome, Italy
Barcelona, 19/2/03