overview of research thrust r3 image and data …agua4 basura cultivo1 cultivo2 cultivo-recortado...
Post on 11-Jul-2020
0 Views
Preview:
TRANSCRIPT
Overview of Research Thrust R3Image and Data Information Management
Overview of Research Thrust R3Image and Data Information Management
Diverse Problems – Similar SolutionsDiverse Problems – Similar Solutions
James ModestinoRPI
Dave KaeliNU
James ModestinoRPI
Dave KaeliNU
R3 Fundamental Research Topics R3a. Data Compression
R3b. Multimedia Networking
R3c. Scalable Computing
R3d. Image Databases
R3 Fundamental Research Topics R3a. Data Compression
R3b. Multimedia Networking
R3c. Scalable Computing
R3d. Image Databases
! Acquisition, transmission, storage, processing, retrieval, dissemination, display and visualization of sensor data
! Development of a distributed collaborative research environment! Provide remote Web access to CenSSIS resources
ScopeScope
SubsurfaceSensing and
Modeling
Validation Testbeds
Environmental-CivilApplications
Bio-MedicalApplications
Image and DataInformation
Management
Physics-BasedSignal Processing andImage Understanding
I–PLUS
R1R1R2R2
R3R3L1L1
L2L2
L3L3
Context and Scope of R3 ResearchContext and Scope of R3 Research
EngineeredSystemEngineeredSystem
EnablingTechnologiesEnablingTechnologies
FundamentalScienceFundamentalScience
Barrier 5Barrier 5
CenSSIS Barriers Addressed in R3CenSSIS Barriers Addressed in R3
Lack of Robust, Physics-Based Recognition and Sensor Fusion TechniquesLack of Robust, Physics-Based Recognition and Sensor Fusion Techniques
Barrier 6Barrier 6 Lack of Rapid Processing and Management of Large Image DatabasesLack of Rapid Processing and Management of Large Image Databases
Lack of Optimal End-to-End Sensor DesignMethodsLack of Optimal End-to-End Sensor DesignMethods
Barrier 7Barrier 7 Lack of Validated, Integrated Processing andComputational ToolsLack of Validated, Integrated Processing andComputational Tools
Barrier 3Barrier 3
Focussed on Attacking Unique CenSSISRequirements for Data Processing and Management
Focussed on Attacking Unique CenSSISRequirements for Data Processing and Management
Goal: Develop information management concepts, tools and infrastructure to support basic CenSSIS scientific mission
Goal: Develop information management concepts, tools and infrastructure to support basic CenSSIS scientific mission
! Distribution
! Transport
! Collaboration
! Web Access
! Distribution
! Transport
! Collaboration
! Web Access
! Representation
! Compression
! Transmission
! Storage
! Representation
! Compression
! Transmission
! Storage
! Computing
! Storage
! Scalability
! Solutionware
! Computing
! Storage
! Scalability
! Solutionware
! Archiving
! Image Formatting
! Image Browsing
! Data Migration
! Archiving
! Image Formatting
! Image Browsing
! Data Migration
Components of R3 Research ThrustComponents of R3 Research Thrust
R3Image and Data
InformationManagement
R3Image and Data
InformationManagement
R3cScalable
Computation
R3cScalable
Computation
R3bMultimedia Networks
R3bMultimedia Networks
R3aData
Compression
R3aData
Compression
R3dImage
Databases
R3dImage
Databases
R3a/bData Compression/
Multimedia Networks
R3a/bData Compression/
Multimedia Networks
R3c/d Scalable Computation/
Image Databases
R3c/d Scalable Computation/
Image Databases
James ModestinoLeader, RPI
Shiv Kalyanaraman, RPIMasoud Salehi, NUJohn Woods, RPIRichard Radke*, RPI
*New Hire-Cost Match
David KaeliLeader, NU
Steve Lerner, WHOIMiriam Leeser, NUWaleed Meleis, NUEric Miller, NU
The R3 Research TeamThe R3 Research Team
Motivation for R3a Data Compression ResearchMotivation for R3a Data Compression Research
" CenSSIS applications will result in massivelylarge data sets.
" Will stress existing processing/storage transmission capabilities.
" Flexible/efficient multiresolution compressionschemes are required.
" General-purpose pixel-based techniques will beinadequate.
" Object-oriented, application-specific approachespromise 1-2 orders of magnitude improvement.
" Reasonable performance/complexity will require exploitation of underlying physics.
Remote HyperspectralSensing of Coastal Regions
UPRM Data Set
Remote HyperspectralSensing of Coastal Regions
UPRM Data Set
MultipleWavelengths
MultipleWavelengths
MultipleImagesMultipleImages
SensorData
SensorData
(MSD)
Wide BandProbe
Narrow BandDetectors
backgroundagua1agua-2agua4basuracultivo1cultivo2cultivo-recortadomontanascultivo4cutlivo-costarenazona-urbanaagua-5cultivo6agua6techo-almacencultivo-corto-Nagua-costa-Nvegetacionrio
Number ofImages/
Wavelength103
Number ofImages/
Wavelength103
Image SizeIn Pixels/View
1K x 1K
Image SizeIn Pixels/View
1K x 1K
Number ofBits/Pixels
16
Number ofBits/Pixels
16
Total SizeOf Data Set20 TBytes
Total SizeOf Data Set20 TBytes
Number ofWavelengths
103
Number ofWavelengths
103
! Transmission Time 56 hr @ 100 Mb/s! Compression Reduces to < 30 min! Utilize 3-D Volumetric Compression! Exploit Correlation Across Wavelengths! Layered Progressive Scheme Desirable
! Transmission Time 56 hr @ 100 Mb/s! Compression Reduces to < 30 min! Utilize 3-D Volumetric Compression! Exploit Correlation Across Wavelengths! Layered Progressive Scheme Desirable
Typical Multi-Spectral Discrimination (MSD) ApplicationTypical Multi-Spectral Discrimination (MSD) Application
Airborne Visible Infrared Imaging Spectrometer (AVIRIS) ApplicationAirborne Visible Infrared Imaging Spectrometer (AVIRIS) Application
! AVIRIS-Visible IR Imager! Multiple Bands
! 224 Bands! 380nm-2500nm
! Individual Images! 397 x 268 pixels! 16 bits/pixel
! Reasonably Large Data Sets! 47.6 Mbytes/SetRemote Hyperspectral
Sensing of Coastal Regions
UPRM Data Set (Band 110)
Multiresolution Approach Based on SPIHTMultiresolution Approach Based on SPIHT
" Multiresolution Wavelet Transform." Exploits Spatial Correlation Properties
of Wavelet Transform." Partitions Data into Spatial Orientation
Tree Sets. " Efficient Strategy for Encoding the
Sorting Information." Efficient Coding of Wavelet Subbands." Embedded Code Providing Lossy-
Lossless Progressive Coding/Transport. " Current Benchmark for Efficient, Low
Complexity Image Encoder." Extensible to 3-D MSD Data Sets Using
3-D Spatial Orientation Trees.2-D SPIHT Frame
Dyadic 3-LevelWavelet SubbandDecomposition
Dyadic 3-LevelWavelet SubbandDecomposition
Comparison of 3-D SPIHT vs. JPEG for AVIRIS ImagesComparison of 3-D SPIHT vs. JPEG for AVIRIS Images
JPEG:0.15 bpp, 24.5 dB
3-D SPIHT:0.15 bpp,29.5 dB
0.5 bpp, 28.2 dB
0.5bpp, 33.4 dB
1.0 bpp, 30.2 dB
1.0 bpp, 35.5 dB
Comparison of 3-D SPIHT vs. JPEG for AVIRIS ImagesComparison of 3-D SPIHT vs. JPEG for AVIRIS Images
JPEG:0.15 bpp, 24.5 dB
3-D SPIHT:0.15 bpp,29.5 dB
0.5 bpp, 28.2 dB
0.5bpp, 33.4 dB
1.0 bpp, 30.2 dB
1.0 bpp, 35.5 dB
Comparison of 3-D SPIHT vs. JPEG for AVIRIS ImagesComparison of 3-D SPIHT vs. JPEG for AVIRIS Images
JPEG:0.15 bpp, 24.5 dB
3-D SPIHT:0.15 bpp,29.5 dB
0.5 bpp, 28.2 dB
0.5bpp, 33.4 dB
1.0 bpp, 30.2 dB
1.0 bpp, 35.5 dB
Comparison of 3-D SPIHT vs. JPEG for AVIRIS ImagesComparison of 3-D SPIHT vs. JPEG for AVIRIS Images
JPEG:0.15 bpp, 24.5 dB
3-D SPIHT:0.15 bpp,29.5 dB
0.5 bpp, 28.2 dB
0.5bpp, 33.4 dB
1.0 bpp, 30.2 dB
1.0 bpp, 35.5 dB
3-D SPIHT vs. JPEG for AVIRIS Images3-D SPIHT vs. JPEG for AVIRIS Images
! Efficient Use of Interband! Information ! Superior R-D Performance
! Approx. 5 dB PSNR! At Least 5x Compress.
! Subjectively Better! No Block Artifacts! Lower Complexity! Embedded/Progressive! Coding! Acquiring and Testing! Other MSD Data Sets! Investigate Other 3-D! Transforms! Use JPEG2000 as Baseline
Good Initial Step in Developing EfficientCompression Schemes for MSD Applications
Good Initial Step in Developing EfficientCompression Schemes for MSD Applications
Focused orPulsed Probe
Focused orGated Detector
(LPM) (MSD)
Wide BandProbe
Narrow BandDetectors
Photomosaic of the Curved Human Retina
RPI Data Set
Photomosaic of the Curved Human Retina
RPI Data Set
MultipleOverlapping
MultipleOverlapping
Temporal Sequence of
Multi-SpectralImages
Temporal Sequence of
Multi-SpectralImages Sensor
DataSensor
Data
A: Retinal Surface (visible channel)B: Subsurface (near infrared)
A: Retinal Surface (visible channel)B: Subsurface (near infrared)
! Real-Time Transmission Requires 720Mb/s ! Compression Reduces to < 6 Mb/s! Transmission Time 6.6 hr @ 100 Mb/s! Utilize Mosaic-Aided Coding Approach ! Physics-Motivated Region-Based Coding! Layered Transport Scheme Desirable
! Real-Time Transmission Requires 720Mb/s ! Compression Reduces to < 6 Mb/s! Transmission Time 6.6 hr @ 100 Mb/s! Utilize Mosaic-Aided Coding Approach ! Physics-Motivated Region-Based Coding! Layered Transport Scheme Desirable
Image SizeIn Pixels1K x 1K
Image SizeIn Pixels1K x 1K
Number ofBits/Pixels
12
Number ofBits/Pixels
12
Total SizeOf Data Set
300-600 GBytes
Total SizeOf Data Set
300-600 GBytes
Length of Procedure
1-2 hrs
Length of Procedure
1-2 hrs
Number ofChannels
2
Number ofChannels
2Frame Rate
30/secFrame Rate
30/sec
Typical LPM/MSD ApplicationTypical LPM/MSD Application
Visible LightSequence
Near IRSequence
Each Frame ofBoth Sequences
Aligned/RegisteredWith Mosaic andWith Each Other
Registration ProcessCan be Used to
Derive Motion Vectorsfor MC PredictionOnly Prediction
Residuals Encoded
Physics-Based Mosaic-Aided Video CodingPhysics-Based Mosaic-Aided Video Coding
Rapid Frame-Frame Motion
PSNR COMPRESSION RATIOS (dB) New Method Baseline
Original Image
52.6 13:1 4:1
47.5 26:1 8:1
42.9 106:1 32:1
38.5 1580:1 128:1
34.00 11000:1 512:1
Decoded Image by New Method after
1580:1 Compression
Decoded Image by New Method after
106:1 Compression
Decoded Image by New Method after
11000:1 Compression
Compression Results for Equal PSNRCompression Results for Equal PSNR
Other R3a Compression WorkOther R3a Compression Work
! Object-Based Compression Schemes! Identify Important Objects in Image! Differential ROI Coding Relative to Background! Matched to Recognition-Based Processing
! Joint Source-Channel Coding (JSCC) Image/VideoTransport Approaches! Joint Design of Source and Channel Coders! Optimum Source/Channel Coding Tradeoffs! Efficient End-to-End Transport of Images and Video
Results Reported in a Number of Reports,Papers and Poster Presentations
Results Reported in a Number of Reports,Papers and Poster Presentations
R3b Multimedia NetworkingR3b Multimedia Networking
CenSSISCenSSIS
backgroundagua1agua-2agua4basuracultivo1cultivo2cultivo-recortadomontanascultivo4cutlivo-costarenazona-urbanaagua-5cultivo6agua6techo-almacencultivo-corto-Nagua-costa-Nvegetacionrio
ClustersClusters CollaborationCollaboration
ConferencingConferencing
Web AccessWeb Access
DatabasesDatabases
Multimedia NetworkInternet 2
Multimedia NetworkInternet 2
TestbedsTestbeds
" Research Issues" Implementation
Internet 2Core Network
Internet 2Core Network
Edge RouterWorkstation
(RPI)Workstation
(NU)Edge Router
" Core Network Typically Overengineered" Bottlenecks at Ingress/Egress Nodes" Keep Core Network Routers Simple" Maintain Single-Class, Best-Effort Delivery Service " Provisioning of QoS Provided Outside Core Network
" Edge-to-Edge (EE) Overlay Architectures" Application Layer Framing (ALF) Using JSCC
" In Either Case, QoS Provisioning Must be ApplicationAware
EEALF
QoS Provisioning for Multimedia TransportQoS Provisioning for Multimedia Transport
155 Mbps
Single Service
CenSSIS
CenSSIS
Internet 2
H.323Conference
Server(RPI)
CenSSIS
CenSSISH.323Clients
H.323ClientsBU
UPRMNU
Evolving CenSSIS Collaborative Work EnvironmentEvolving CenSSIS Collaborative Work Environment
Will Evolve From H.323 Videoconferencing Model with Extensions
Will Evolve From H.323 Videoconferencing Model with Extensions
OtherCollaborators
CenSSIS
H.323 IP Telephony
Standard
RPI
Status of Collaborative Work EnvironmentStatus of Collaborative Work Environment
! Internet 2 Connectivity Between RPI and NU.
! Remote Access to Compressed Image Database at NU Underway.
! Several Collaborative Applications Under Study:! Delivery of CenSSIS-Related Courses! High-Quality Video Conferencing! Remote Access to Testbeds
! Incorporate Multimedia Networking ResearchResults.
R3c/d Research ProjectsR3c/d Research ProjectsR3c. Parallel Computing
• Beowulf Cluster Development (C. Shaffer and D. Kaeli) • Profile Guidance and Characterization Tools of Data Parallel Applications (D. Kaeli and Mercury Computer)• Parallel Programming Tools Targeting CenSSIS Applications (W.Meleis, D. Kaeli, D. Brooks, C. Rappaport, M. El-Shenawee and ISI)
R3c. Reconfigurable Embedded Computing• Applying Reconfigurable Hardware to Segmentation for MultispectralImagery (M. Leeser and Los Alamos) • An Investigation into an FPGA Implementation of Backprojection for Rapid Topographic Imaging (E. Miller, M. Leeser and Mercury )
R3d. Image Databases and Metadata• Content Searchable Image Databases (R. Norum, D. Kaeli, E. Miller, S. Lerner and J. Howland)
How can CenSSIS applications scale up to exploit supercomputers to overcome processing barriers?
! Provide a pathway for researchers to exploit NSF Partnerships for Advanced Computational Infrastructure (PACI) resources
! Construct Beowulf-class clusters to provide low-cost, scalable processing (BioMed 8-node cluster and NSF MRI 32-node cluster)
! Utilize BU Mariner SGI clusters! Develop parallel programming toolsets around MPI,
MATLAB and Java standards
How do we parallelize CenSSIS applications?
! CenSSIS applications possess unique processing characteristics
! We have developed a methodology specifically tailored to the data-dominated nature of CenSSIS applications
! We consider parallelism at three grains:! Coarse-grained (potentially application-level)! Loop-grained! Data parallelism
Main0/2502
Multdibobj2/25
Tfqmr1/1604
3/35
Mult11/1602Multdibso
4/59
Mult4/1599
Multdib50/680
Diflinit0/852
Multdibos3/27
Multfaflone4m523/6
Multfaflone4359/5
Multdifl706/0
Snuin7121/84
Epsumag290/94
Epsued42/637Suin7
29/227
0/0
0/0
0/0
18/0
16/0Function
Self /Children Runtime
FunctionSelf /Children Runtime
Number of Calls
0/0 0/0
1
1018
1018
1018
1018
14.4 M
14.4 M14.4 M
14.4 M
11
11
273
273
273
273
273
273
273
54K2.4M
0.6M
Soil
Air
Mine
Steepest Descent Fast Multi-pole Method (SDFMM) Program Control FlowSteepest Descent Fast Multi-pole Method (SDFMM) Program Control Flow
Soil
Air
Mine
Program run time versus number of processors
0500
10001500200025003000350040004500
0 5 10 15 20 25 30
Number of processors
Run
time
(s)
The w hole run timeThe unparallelized part
Fine-grained and coarse-grainedparallelization using MPI on a 32-node clusterFine-grained and coarse-grainedparallelization using MPI on a 32-node cluster
Speedup from parallelization of main loop
05
101520253035
0 5 10 15 20 25 30 35
Number of Processors
Sp
eed
up
CenSSIS Parallel Processing
Status! Obtained linear speedup on SDFMM ! Developed a complete methodology for
assessing control and data parallelism! Completing construction of next generation
gigabit-based cluster
Year 2 Goals! Fully exploit availability of clusters and PACI! resources! Parallelize FDTD, FDFD and SAMM
algorithms
Image ID Image typeLPM099 HyperspectralMVT002 ECGMSD321 Diffusive waveMVT061 CAT scan
y = C(x, z) + n
Database (Oracle8i)
Metadata
* TBs of RAID storage
* Replication and distribution
* Security
*Image format standards
! The data about the data! Contextual browsing
How do we enable researchers to share their image/sensor data?How do we enable researchers to share their image/sensor data?
Sensor Manufacturer Calibration date
Serial number
Compression algorithm
Software Version
Related publications
HLLs supported
CenSSIS Image DatabasesCenSSIS Image Databases
Example of XML labelling
<?xml version="1.0" encoding="UTF-8" ?>
<embryo>
<feature label="1" xPos1 ="50" yPos1="50"xPos2="65" yPos2="65"> Nucleus </feature>
<feature label="2" xPos1="75" yPos1="75" xPos2="85" yPos2="85"> <fragmentation>none</fragmentation> </feature>
<feature label="3" xPos1="100" yPos1="10" xPos2="110" yPos2="15"> <description>mouse oocyte</description> </feature>
<feature label="4" xPos1="25" yPos1="25" xPos2="35" yPos2="35">Cell Membrane </feature>
</embryo>
nucleus
12
34
12
34
mouse oocyte
CenSSIS Image DatabasesCenSSIS Image Databases
Status! Submission, query and update capabilities
online! Includes both images and image metadata! Basic authentication for website
implemented! Construction in progress of the CenSSIS
database/fileserver cluster (Sun)
Year 2 Goals! Implement secure web authentication (SSL)! XML image feature tagging! XML query capabilities
R3 Interaction with IndustryR3 Interaction with Industry
! Mercury Computer! Porting CenSSIS applications to RACE++! Profile-guided compilation! Back projection acceleration with FPGAs
! Integrated Sensors! RTExpress - parallel MATLAB
! MathWorks! Object-oriented MATLAB
! Future partners! EMC - storage array! Clinomics - drug interaction analysis using
content-based image indexing
! Mercury Computer! Porting CenSSIS applications to RACE++! Profile-guided compilation! Back projection acceleration with FPGAs
! Integrated Sensors! RTExpress - parallel MATLAB
! MathWorks! Object-oriented MATLAB
! Future partners! EMC - storage array! Clinomics - drug interaction analysis using
content-based image indexing
Relationships with Other Research ThrustsRelationships with Other Research Thrusts
R1SDFMM, FTFT, FTDT,
SAMM Models Multi-sensor instruments
BioBED, MedBED, SeaBED, SoilBEDMotivating applications +
experimental data
R3Parallel computation
Image databasesSoftware toolsCompressionNetworking
Visualization
L1
L2
R2Inverse ECG solutionsSub-retinal mapping
Diffuse optical tomographyHyperspectral imaging
What are our plans for Year 2?What are our plans for Year 2?
! Physics-Based Image and Data Compression! 3-D Volumetric Compression of MSD Data! Mosaic-Aided Compression of Retinal Sequences! Incorporate Region-of-Interest (ROI) Coding
! Multimedia Networking! QoS Provisioning on Internet 2! Development of Collaborative Work Environment
! Parallelization Processing! Tools for a broad set of R1 modeling methods including
FDTD, FTFD and SAMM! Utilization of RTExpress and parallel MATLAB
! Image Databases! Content-based XML query capability of image data! Multi-dimensional database indexing and visualization
! Physics-Based Image and Data Compression! 3-D Volumetric Compression of MSD Data! Mosaic-Aided Compression of Retinal Sequences! Incorporate Region-of-Interest (ROI) Coding
! Multimedia Networking! QoS Provisioning on Internet 2! Development of Collaborative Work Environment
! Parallelization Processing! Tools for a broad set of R1 modeling methods including
FDTD, FTFD and SAMM! Utilization of RTExpress and parallel MATLAB
! Image Databases! Content-based XML query capability of image data! Multi-dimensional database indexing and visualization
top related