sensor fusion laboratory
DESCRIPTION
MISSION: Study the benefits of using simultaneous information from multiple sensors to probe the environment. SENSOR FUSION LABORATORY. Thad Roppel, Associate Professor AU Electrical and Computer Engineering Dept. [email protected]. EXAMPLES - PowerPoint PPT PresentationTRANSCRIPT
SENSOR FUSION LABORATORY
Thad Roppel, Associate Professor AU Electrical and Computer Engineering Dept.
EXAMPLES• Infrared / Millimeter wave radar for vehicle detection and
identification• Chemical sensor arrays – “artificial nose”• Biomimetics – imitating animal sensorimotor behaviors• Biomedical – using electrical and optical probes to study
cardiac arrhythmias
MISSION: Study the benefits of using simultaneous information from multiple sensors to probe the environment.
Personnel and PublicationsPERSONNEL•Ting-To Lo (PhD): Molecular Switching in Biosensors•Rama Narendran (PhD): Biomimetic Simulations of Organized Machine
Behavior•Jun Pan (PhD): Wireless Protocol for Electrical and Optical Cardiac Microprobes•Aroldo Couto (MS): Flight Stabilization Using Adaptive Artificial Neural
Networks•Brian Wingfield (MS): Silicon Processing for Lateral Emission Fiber-Optic
SensorsREPRESENTATIVE RECENT PUBLICATIONS• D. M. Wilson, T. Roppel, and R. Kalim, "Aggregation of Sensory Input for
Robust Performance in Chemical Sensing Microsystems," Sensors and Actuators B, 64(1–3), 107-117, June 2000.
• T. Roppel and D. M. Wilson, "Biologically-Inspired Pattern Recognition for Odor Detection," Pattern Recognition Letters, 21(3), 213–219, March 2000.
• D. M. Wilson, K. Dunman, T. Roppel, and R. Kalim, "Rank Extraction in Tin-Oxide Sensor Arrays," Sensors and Actuators B, 62(3), 199-210, April 2000.
• T. Roppel, R. Kalim, and D. Wilson, "Sensory Plane Analog-VLSI for Interfacing Sensor Arrays to Neural Networks, " Virtual Intelligence and Dynamic Neural Networks VI-DYNN '98, Stockholm, Sweden, June 22-26, 1998.
IR / MMW DATA FUSIONSupport: AFOSR 1992-93
Project Goal: Improved identification of military vehicles from aerial scenes.
LANCE Missile Launcher
T-62 Tank
M-113 Armored Personnel Carrier (APC)
IR / MMW Fusion, cont’dAPPROACH:
IR SCENE PIXELS
MMW RADAR DATA
NEURAL NETWORK
APCTANKLAUNCHER
PERFORMANCE ASSESSMENT: A T LA + - -T - + -L - - +
•Multiple permutations
•Confusion matrix•Average result
OVERALL RESULT: 14 % improvement with sensor fusion
Chemical Sensor ArraysSupport: DARPA 1997-99
PROJECT GOAL: Improved identification and detection of chemical plumes in non-laboratory conditions.
VEHICLESENSORS
PLUME COMMANDSTATION
RF LINK
ROAD
WIND
Canine Training at IBDSAuburn is world-renowned for training of detection dogs at the Institute for Biological Detection Systems.
Chemical Sensor Arrays, cont’d
Odor Sensor Array
0 100 200 300 400 5000
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
Timestep
Sen
sor V
olta
ge
Sensor Outputs
Sensor Array Dynamic Response
Chemical Sensor Arrays, cont’d
0 100 200 300 400 5000
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
Timestep
Sen
sor V
olta
ge
10 20 30 40 50
2468
1012
14
Sen
sor N
umbe
rTimestep
Sensors 1-15
Raw Output Thresholded Binary Output
Above ThresholdBelow ThresholdPreprocessing
Chemical Sensor Arrays, cont’dac
eSample 1 Sample 2
1
20
Sample 3 1
20
amm
dal
g87
g89
g93
oil
pth
Sensor #
xyl
5 10 15Sensor #5 10 15
Sensor #5 10 15
Chemical Sensor Arrays, cont’d
input categories
netw
ork
resp
onse 1 timestep
aceammdalg87g89g93oilpthxyl
5 timesteps 10 timestepsne
twor
k re
spon
se 20 timestepsaceammdalg87g89g93oilpthxyl
50 timesteps Ideal Response
Time Evolution of Confusion Matrix: Forward SequenceTrained for 20 timesteps
00.10.20.30.40.50.60.70.80.91
Chemical Sensor Arrays - Summary
A recurrent neural network was trained to recognize 9 odors presented in an arbitrary time sequence.
Response time is reduced by an order of magnitude by threshold preprocessing.
Well-suited for use as a front-end for a hierarchical suite of NN’s in a portable, near-real time odor classification device.
BIOMIMETICSSupport: Under discussion with AF Advanced Guidance Division, Munitions Directorate at Eglin AFB
PROJECT GOAL: Learn sensor fusion from animals. Apply this to flying a drone to target using onboard video.
Flies land accurately
Bees find flowers
Bats catch evading insects in flight
BIOMIMETICS, cont’d
What do they “know” that we don’t?One possibility is that they use variations of optic flow.
Represent sensory image field by motion vector field.
Image Sequence
Optic Flow Field
BIOMIMETICS, cont’d
EXAMPLESA fly can land simply by maintaining constant optic flow.A dog can track by maintaining constant sensory flow across olfactory epithelium and following the gradient (using sniffing as a form of “chopper amplifier.”
Question to be answered: Can we guide a missile to target with a similar approach?
END OF PRESENTATION