t opic 5: l evel 0 david l. hall. t opic o bjectives introduce concepts of level 0 processing ...
Post on 30-Dec-2015
216 Views
Preview:
TRANSCRIPT
TOPIC 5: LEVEL 0
David L. Hall
TOPIC OBJECTIVES
Introduce concepts of level 0 processing Identify categories of techniques for level 0
processing (e.g., signal and image processing) Extend the concept of traditional signal and
image processing and conditioning to meta-data generation
LEVEL 0 PROCESSING
COMMENTS ON LEVEL 0 PROCESSING
Level 0 processing concerns processing each source of data independently to obtain the most useful information possible; we want to “squeeze out” useful information
In general, input data to a fusion system may include scalar or vector data, signal data, image data, or textual information. Each of these classes of information may entail an entire range of potentially applicable techniques (e.g., signal processing, image processing, text-based processing)
It is beyond the scope of this lecture (and indeed this entire course) to address these subjects – however, we seek to make the student aware of the need to consider such processing
Finally, an emerging new area is the generation of semantic meta-data (e.g., automatic semantic labeling of images) as a new powerful process to improve data fusion.
METASENSOR PROCESSING
METASENSOR PROCESSING
CA
TEG
OR
YFU
NC
TIO
N
TEC
HN
I QU
E
Sensor Validation
and Calibration
Parametric
Modeling
• Spectral analysis• Wavelet processing• Short-time Fourier transform• AR modeling• System I/D analysis
Entity
Detection
• Array processing - Synthetic aperture (SA) - Inverse SA• Moving target indicator (MTI) • Hypothesis testing• Pre-detection fusion• Distributed fusion
• Phase and gain calibration• Sensor data validation
Non-Parametric
Analysis
• Time-series analysis• Dynamical systems analysis• Image analysis• Neural networks• Trend analysis• Thresholding and integration• Energy detection
LEVEL 0 PROCESSING BY SOURCE TYPE
Scalar orVectorSensor
Signalsensor
ImagingSensor
Text-based information
yt
Semantic processing
Signal processing
Image processing
Data Conditioning
Original Source Data plus transformed & added meta-data
SOURCE TYPE: SCALAR & VECTOR
DATA Representative techniques
Scaling methods and transformations Coordinate transformations and rotations Smoothing, filtering, averaging Thresholding Feature extraction and representations
y y + z Transformation
Original data Original data Alternate representations
SOURCE TYPE: SIGNAL DATA Representative Parametric
Modeling Techniques Spectral analysis Wavelet processing Short-time Fourier transform Auto-Regression modeling System Identification analysis
Representative Non-Parametric Modeling Techniques
Time-series analysis Dynamical systems analysis Neural networks Trend analysis Threshold & integration Energy detection
Transform + characterization vector, y
Original signalTransformed signal
EXAMPLES OF SIGNAL PROCESSING
Signal processing and enhancement are standard on most radios & receivers High & low frequency filtering Boost of selected frequency ranges “Coloring” of signals
Active Noise Cancelation devices Active cancellation of background
noise (e.g., airline engine noise)
Extensive commercial tools are available such as MATLAB, Mathematica, etc
SOURCE TYPE: IMAGE DATA Classification Feature Extraction Pattern Recognition Projection Multi-Scale Signal Analysis
Principal Components Analysis Independent Component Analysis Self-Organizing Maps Hidden Markov Models Neural Nets
+ Meta-Data
Original Image Transformed image
ImageProcessing
An enormous amount of methods exist to process and transform image data from the individual pixel level, to object level to complete image level. Commercial tools such as Photoshop Pro make many of these available to casual users
EXAMPLES OF IMAGE PROCESSING
http://web.uct.ac.za/depts/physics/laser/hanbury/intro_ip.html
Contrast enhancement
Image sharpening using wavelets
http://www.phasespace.com.au/wavelet_ex.htm
Deconvolution
http://www.phasespace.com.au/wavelet_ex.htm
SOURCE TYPE: TEXT DATA
Representative techniques Key word extraction Name disambiguation Link of semantics to parametric data (e.g. location) Syntactic processing Thesaurus processing Semantic distance calculations Links to other documents/
Textual input via human reports or web info.
Textual input via human reports or web info.
Meta Data - key words - lexicon data - identified parameters - links to related documents
+TextualProcessing
CONCEPT OF SEMANTIC META DATA GENERATION
Signal & images represent a data level view of patterns, features & characteristics
In human pattern recognition, we identify labels to represent signals & data
New techniques are being developed to train computers to perform a similar function
PROBLEM: HOW TO EFFECTIVELY QUERY ON A VERY LARGE COLLECTION OF SATELLITE IMAGERY FOR
SEMANTICALLY MEANINGFUL REGIONS ?
Use semantic categorization combined with CBIR on small “patches” of the images (Dr. James Wang)
QUERY FORMULATION
Query “patch” – pertaining to some semantics, e.g. mountains
Satellite Image Database Ranked
ResultsPurpose ?
Geography - Find mountainous regions with snow-caps (low-level semantics).
Forestry – Find forests of a certain density, analyze deforestation (mid-level semantics).
Military – Find air-bases in certain regions of the world (high-level semantics).
LIMITATIONS OF CURRENT APPROACHES
Difficulty in defining classes Continuum of variety Subjectivity
No major improvement in classification accuracy G. G. Wilkinson, “Results and Implications of a Study of Fifteen Years of
Satellite Image Classification Experiments,” IEEE Trans. On Geoscience and Remote Sensing, 2005.
Recognition of higher-order semantics More focus on spectral than spatial dimensions
Scalability of complex querying and browsing
Flexibility to handle diverse applications
AUTOMATIC SEMANTIC CATEGORIZATION
SUMMARY OF APPROACH
Practical implementation on large-scale archives
Flexibility to adapt to various applications of satellite imagery, e.g., Geography, Military, Metallurgy, Agriculture etc.
Exploiting spectral and spatial information using a generative model for semantic categorization (supervised learning)
Handling of untrained classes (supervised learning)
Using a scalable CBIR system for efficient querying and browsing (unsupervised)
OVERVIEW OF THE SYSTEM
Classification Generative Classifier: Two-dimensional Multi-resolution Hidden Markov Models
(2-D MHMM) Handling untrained classes
Discriminative Classifier: Support Vector Machines Querying for fast retrieval
Integrated Region Matching (IRM) measure used in SIMPLIcity system
K trained 2D-MHMMs
40 x K training patches
Query patch
Patch Class
Random samples from trained and untrained classes
K Likelihood scores Trained
biased SVM
Database search using IRM
Query Results
DETAILS OF THE ARCHITECTURE
Efficient database design
Pre-computation of classes using 2D-MHMM for faster response
Suitable user interface and visualization (Common CBIR issue)
TRAINING PROCESS
AUTOMATIC SEMANTIC CATEGORIZATION
RETRIEVAL RESULTS
http://riemann.ist.psu.edu/image
RETRIEVAL RESULTS
http://riemann.ist.psu.edu/image
TOPIC 5 ASSIGNMENTS
Preview the on-line topic 5 materials Read Farid paper (2008)
DATA FUSION TIP OF THE WEEK
Level 0 processing is a vital part of the data fusion process - in essence pre-processing or conditioning data from individual sources and sensors. This kicks off all subsequent processing. It is necessary to do the best job possible at this stage without either introducing spurious information into the observed data, nor failing to extract and enhance the data “at the source”. It is not possible to use “down-stream” data fusion processing to make up for failures at the level 0 stage.
top related