image registration advanced dip project group #3: dave grimm joe handfield mahnaz mohammadi yushan...
Post on 18-Jan-2016
225 Views
Preview:
TRANSCRIPT
Image Image RegistrationRegistration
Advanced DIP ProjectAdvanced DIP ProjectGroup #3:Group #3:
Dave GrimmDave Grimm
Joe HandfieldJoe Handfield
Mahnaz MohammadiMahnaz Mohammadi
Yushan ZhuYushan Zhu
OutlineOutline Image RegistrationImage Registration Point MappingPoint Mapping Problem StatementProblem Statement GCP Selection MethodsGCP Selection Methods
ManualManual Contour Based GCPContour Based GCP Corner and Edge BasedCorner and Edge Based
Evaluation MethodsEvaluation Methods Group PlanGroup Plan
Time TableTime Table Responsibilities Responsibilities
Image RegistrationImage Registration Spatial matching the pixels of two (or Spatial matching the pixels of two (or
more) images of the same area or scenemore) images of the same area or scene
Relates the geometric coordinate system in Relates the geometric coordinate system in one image to anotherone image to another
Transforms one of the images so that the Transforms one of the images so that the two images share a common coordinate two images share a common coordinate system system
Image Registration Image Registration (Cont.)(Cont.)
Applications of Image Applications of Image RegistrationRegistration
Remote SensingRemote Sensing Extract information from images of the same Extract information from images of the same
region taken at different times or in different region taken at different times or in different spectral bandsspectral bands
Color ScienceColor Science Creating a mutispectral imagesCreating a mutispectral images
MedicalMedical Pathology analysisPathology analysis
Image FusionImage Fusion Image Mosaicking Image Mosaicking
Point MappingPoint Mapping
Widely usedWidely used Standard technique for registering images Standard technique for registering images
misaligned by an misaligned by an unknownunknown transformation transformation
Requires ground control points (GCPs Requires ground control points (GCPs or primitives) to be found in the imagesor primitives) to be found in the images Intrinsic or extrinsicIntrinsic or extrinsic Can be done either manually or Can be done either manually or
automaticallyautomatically
Point Mapping (cont)Point Mapping (cont) Mathematically relates the coordinate Mathematically relates the coordinate
systems of the imagessystems of the images
aa00 and b and b00 are needed for a simple shift of origin, are needed for a simple shift of origin, and the first two terms are needed for a and the first two terms are needed for a combined scale adjustment and shifting of the combined scale adjustment and shifting of the originorigin
Higher order equations for more complicated Higher order equations for more complicated transforms are possible, such as rotation, skew, transforms are possible, such as rotation, skew, and perspective differencesand perspective differences
x a0 a1 x a2 y a3 x y a4 x 2 a5 y 2...x
y b0 b1 y b2 x b3 y x b4 y 2 b5 x 2 ...y
Point Mapping StepsPoint Mapping Steps Select GCPs from Select GCPs from
each imageeach image Match GCPs to Match GCPs to
from point pairs from point pairs (points that are (points that are spatially the same spatially the same in the two images)in the two images)
Register images Register images via point mappingvia point mapping
Our focus is on Our focus is on Steps 1 & 2Steps 1 & 2
Problem StatementProblem Statement Comparison of GCP selection algorithmsComparison of GCP selection algorithms
ManualManual
AutomatedAutomated Area-basedArea-based Feature-basedFeature-based
Contour MappingContour Mapping Corner and Edge DetectionCorner and Edge Detection
Manual RegistrationManual Registration
Left: the reference image Right: the image to be registered
Automated RegistrationAutomated Registration Area-based algorithmsArea-based algorithms
A small window of points in the sensed image, A small window of points in the sensed image, correlation kernel, is compared statistically with correlation kernel, is compared statistically with windows of the same size in the reference image. windows of the same size in the reference image. The measure of similarity is usually the normalized The measure of similarity is usually the normalized cross correlation. The location of maximum in the cross correlation. The location of maximum in the normalized correlation image is a pair of GCP. normalized correlation image is a pair of GCP.
Disadvantage : the correlation value is sensitive to scale Disadvantage : the correlation value is sensitive to scale and rotationand rotation
g i, j 1
K1K2
f i, j h i, j
Automated Registration Automated Registration (Cont.)(Cont.)
Feature-based algorithmFeature-based algorithm Spatial features usually include edges, Spatial features usually include edges,
boundaries, intersections, etc boundaries, intersections, etc The general feature representations The general feature representations
are:are: Chain codeChain code Moment invariantsMoment invariants Fourier descriptorFourier descriptor Shape signaturesShape signatures
Advantage: invariant to scaling, Advantage: invariant to scaling, rotation, and translationrotation, and translation
Feature- based algorithmFeature- based algorithmContour- basedContour- based
Image segmentationImage segmentation
Image matchingImage matching
Image SegmentationImage Segmentation
Producing closed-edged contours by Producing closed-edged contours by convolving the original image with Laplacian convolving the original image with Laplacian of Gaussian (LoG) operatorof Gaussian (LoG) operator Find zero crossing points, in which the convolved Find zero crossing points, in which the convolved
image is scanned to detect pixels that have zero image is scanned to detect pixels that have zero value or pixels at which a change of sign has value or pixels at which a change of sign has occurred. Staring with a pixel, its neighboring occurred. Staring with a pixel, its neighboring pixels were expanded until a sign change occurred.pixels were expanded until a sign change occurred.
Drawbacks:Drawbacks: Discontinuity at the weak edge pixelsDiscontinuity at the weak edge pixels Thick edgesThick edges
Image segmentation (Cont.)Image segmentation (Cont.)Thin and Robust Zero- Thin and Robust Zero-
CrossingCrossing
Mark as an edge point every pixel Mark as an edge point every pixel that satisfies the following conditions:that satisfies the following conditions: The pixel is a zero-crossing pointThe pixel is a zero-crossing point The pixel lies in the direction of the The pixel lies in the direction of the
steepest gradient change (edge strength)steepest gradient change (edge strength) The pixel is the closest pixel to the virtual The pixel is the closest pixel to the virtual
zero plane of the LoG image among its zero plane of the LoG image among its eight neighborseight neighbors
Image Image SegmentationSegmentation
Thin and Robust Thin and Robust Zero- CrossingZero- Crossing
(Cont.)(Cont.)
Discarded the noisy edge Discarded the noisy edge points using points using Edge sorting Edge sorting
Edge refinementEdge refinement
Image MatchingImage Matching
Invariant momentInvariant moment Produce a set of scaled moment-based Produce a set of scaled moment-based
descriptor of planar shapes, that are descriptor of planar shapes, that are scale, rotation, and translation invariantscale, rotation, and translation invariant
Improved chain-coded Improved chain-coded representation of regionsrepresentation of regions
Image Matching (Cont.)Image Matching (Cont.)Chain codingChain coding
A way to represent a boundary by a A way to represent a boundary by a connected sequence of straight-line connected sequence of straight-line segments of specific length and segments of specific length and direction based on 4- or 8- direction based on 4- or 8- connectivityconnectivity
Image Matching (Cont.) Image Matching (Cont.) Draw backs of Standard Draw backs of Standard
Chain CodingChain Coding
The resulting chain codes tend to be The resulting chain codes tend to be quite longquite long
Any small disturbance along Any small disturbance along boundary due to noise or imperfect boundary due to noise or imperfect segmentation causes changes in the segmentation causes changes in the code that may not be related to the code that may not be related to the shape of the boundaryshape of the boundary
Improved Improved Chain-CodeChain-Code
1.1. Shift Operation:Shift Operation:
2.2. Smoothing: GaussianSmoothing: Gaussian3.3. Normalization: Normalization:
Demean Demean 4.4. Resampling OperationResampling Operation
minimized is
08mod)(int,
,
1
11
ii
iii
ii
bq
aqq
qbab
Image Matching (Cont.)Image Matching (Cont.)
Invariant-Moment Distance MatrixInvariant-Moment Distance Matrix
The pairs are accepted as candidate The pairs are accepted as candidate matches if their invariant-moment matches if their invariant-moment values are below the defined values are below the defined thresholdsthresholds
dij rk i s
k i 2
k1
7
Image Matching (Cont.)Image Matching (Cont.)
Chain-code Matching MatrixChain-code Matching MatrixContour A and B selected as matched pair if:
1) DAB≥DAB’ where B’ includes all the contours with similar shapes to A
2) DAB≥T3 where T3 is a preset threshold which can eliminate matches with poor correlation
Image Matching (Cont.)Image Matching (Cont.)
The smaller the Invariant-Moment The smaller the Invariant-Moment distance, the more similar the shapes distance, the more similar the shapes of two regionof two region
The greater the chain-code matching The greater the chain-code matching coefficient, the more contours coefficient, the more contours resemble each other in the shaperesemble each other in the shape when Dwhen Dklkl=1, there is a perfect match=1, there is a perfect match
The centroid of the matched contours The centroid of the matched contours are used as GCPsare used as GCPs
Summary of Contour Based Summary of Contour Based AlgorithmAlgorithm
Corner and Edge Corner and Edge DetectionDetection
The corners and The corners and edges present in each edges present in each image are locatedimage are located Harris and Stephens, Harris and Stephens,
19881988
A local window is A local window is placed in the image and placed in the image and changes due to shifting changes due to shifting the window are the window are consideredconsidered
Corner and Edge Detection Corner and Edge Detection (cont)(cont)
An edge produces large changes An edge produces large changes when the window is shifted when the window is shifted perpendicular to the edge direction perpendicular to the edge direction and small changes when shifted and small changes when shifted parallel parallel
A corner produces large changes A corner produces large changes when the window is shifted either when the window is shifted either perpendicular or parallelperpendicular or parallel
Insignificant points (noise) are Insignificant points (noise) are removed via thresholdingremoved via thresholding
Corner and Edge Detection Corner and Edge Detection (cont)(cont)
Each found point in the image to be registered Each found point in the image to be registered (image B) is then compared to each found point (image B) is then compared to each found point in the reference image (image A) to determine in the reference image (image A) to determine which pairs matchwhich pairs match Any point without a match is considered an outlier Any point without a match is considered an outlier
(slack)(slack)
Test ImagesTest Images
Images that we can introduce known Images that we can introduce known transformationstransformations
Various images with unknown Various images with unknown transformationstransformations
Limit testing to grayscale images for nowLimit testing to grayscale images for now
Statistical Assessment of Statistical Assessment of Image Registration Image Registration
ResultsResults
Assessment would be done on Assessment would be done on unknown GCP pairs using calculated unknown GCP pairs using calculated transformation matrix obtained by transformation matrix obtained by different point mapping algorithmsdifferent point mapping algorithms
Statistical AssessmentStatistical Assessment
Calculation of statistical distance for Calculation of statistical distance for each pair of points, P(x1,y1), each pair of points, P(x1,y1), Q(x2,y2)Q(x2,y2)
d P,Q x1 x2 2
s11
y1 y2 2
s22
P x1, y1 &
Q x2, y2
Sik 1
nx ji x x jk x k
j1
n
i 1,2,..., p k 1,2,..., p
X x1 y1
x2 y2
Statistical AssessmentStatistical Assessment
Take maximum, standard deviation, Take maximum, standard deviation, and average error between the group and average error between the group of GCPs using different of GCPs using different transformation matrixtransformation matrix
Evaluate variability of predicted Evaluate variability of predicted points along x and y axis using scatter points along x and y axis using scatter plotplot
Statistical AssessmentStatistical Assessment
Calculate the confidence intervalsCalculate the confidence intervals
H0 : 0 versus H1 : o
x i tn 1 2 sii
nii x i tn 1 2 sii
n
Registration Time TableRegistration Time Table
Week 4 – Plan presentation, programmingWeek 4 – Plan presentation, programming
Week 5 – ProgrammingWeek 5 – Programming
Week 6 – At least manual code done, present Week 6 – At least manual code done, present preliminary preliminary resultsresults
Week 7 – Evaluation methods done Week 7 – Evaluation methods done
Week 8 – Both automated methods finishedWeek 8 – Both automated methods finished
Week 9 – Gathering image results, complete write-Week 9 – Gathering image results, complete write-upup
Week 10 – Final presentation & reportWeek 10 – Final presentation & report
Plan FlowchartPlan Flowchart
Manual
Contour
C&E
Point-Mapping Registration Algorithm
Methods:
Correlations
Confidence Interval
Evaluation:Method flow:
GCPs
Task Assignment ListTask Assignment List
Joe - Manual pixel detection methodJoe - Manual pixel detection method
Yushan - Image contour detection Yushan - Image contour detection method method
Dave - Corner and edge detection Dave - Corner and edge detection
Mahnaz - Statistical evaluation (with Mahnaz - Statistical evaluation (with Joe)Joe)
top related