Visual and Infrared Patch-wise DCT based
Image Fusion using Artificial Bee Colony
Optimization and Image Segmentation
Thesis submitted in the partial fulfillment of the requirements for
The award of the degree of
MASTER OF ELECTRICAL ENGINEERING
Submitted By
DEBASIS MAJI
Registration number: 120904 of 12-13
Examination Roll number:M4ELE14-10
Under Guidance of
Prof. Samar Bhattacharya
Electrical Engineering Department
Faculty Council of Engineering and Technology
JADAVPUR UNIVERSITY
KOLKATA-700032
YEAR - 2014
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
ii
Faculty Council of Engineering and Technology
JADAVPUR UNIVERSITY
KOLKATA-700032
Certificate of Recommendation
This is to certify that Mr. DEBASIS MAJI (M4ELE14-10) has completed his dissertation
entitled An approach towards History based Visual and Infrared patch-wise DCT based image
fusion using Artificial Bee Colony Optimization and image segmentation, underthe direct
supervision and guidance of prof. Samar Bhattacharya, Electrical Engineering Department,
Jadavpur University. We are satisfied with the work, which is being presented for the partial
fulfillment of the degree of Master of Electrical Engineering of Jadavpur University, Kolkata-
700032.
----------------------------------------
Prof. Samar Bhattacharya
Professor, Electrical Engineering Department
Jadavpur University, Kolkata-700032
_______________________________
_______________________________
Prof. Samar Bhattacharya Head of the Department
Department of Electrical Engineering
Jadavpur University, Kolkata-700032
Prof. Sivaji Bandyopadhyay Dean,
Faculty Council of Engineering and Technology
Jadavpur University, Kolkata-700032
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
iii
Faculty Council of Engineering and Technology
JADAVPUR UNIVERSITY
KOLKATA-700032
Certificate of Approval *
The foregoing thesis is hereby approved as a creditable study of Master of Electrical Engineering
and presented in a manner satisfactory to warrant its acceptance as a prerequisite to the degree
for which it has been submitted. It is understood that by this approval the undersigned do not
necessarily endorse or approve any statement made, opinion expressed or conclusion therein but
approve thesis only for the purpose it is submitted.
Final Examination for
Evaluation of the Thesis ------------------------------------
--------------------------------------
(Signature of Examiners)
*Only in case the thesis is approved
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
iv
Declaration of Originality and Compliance of Academic Ethics
I hereby declare that this thesis contains literature survey and original research work by the
undersigned candidate, as part of his Master of Electrical Engineering.
All information in this document has been obtained and presented in accordance with academic
rules and ethical conduct.
I also declare that, as required by these rules and conduct, I have fully cited and referenced all
material and results that are not original to this work.
Name (Block Letters) : DEBASIS MAJI
Exam Roll Number :M4ELE14-10
Thesis Title :Visual and Infrared patch-wise DCT based image fusion
Using Artificial Bee Colony Optimization and Image
Segmentation.
Signature with Date :
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
v
Dedicated
To
My Parents,
Who have always supported me in all myWho have always supported me in all myWho have always supported me in all myWho have always supported me in all my
Endeavors Endeavors Endeavors Endeavors …….…….…….…….
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
vi
ACKNOWLEDGEMENTSACKNOWLEDGEMENTSACKNOWLEDGEMENTSACKNOWLEDGEMENTS
It is a pleasant task to express my gratitude to all those who have accompanied and helped me in
my thesis.
First and foremost, I really take this opportunity to express my deep sense of gratitude to my
guide, Prof. Samar Bhattacharya, Department of Electrical Engineering, Jadavpur University,
Kolkata, for his invaluable guidance, suggestions encouragement throughout the project which
helped me a lot to improve this project work. It has been very nice to be under his guidance. His
appreciation during the good times has been boosting my morals. I have been extremely lucky to
have him as my guide.
I am also thankful to Prof. Samar Bhattacharya, Smt. MadhubantiMaitra and Dr. R. K. Barai
for their guidance in the seminar classes.
I would also like to convey my gratitude to Prof. AmitavaChatterjee, Gourhari Das and Dr. Smita
Sadhu for their encouragement and valuable suggestions throughout the course of this work.
I am also indebted to Prof. Samar Bhattacharya, the Head of the Department of Electrical
Engineering, and co-operation during this thesis work.
My heart-felt thanks also goes to all my family members for their love and encouragement,
without which, the work would not have been possible.
Last, but not the least, I would like to thank my batch-mates, who have directly or indirectly
helped me in this work.
Date: _______________ ______________________
Jadavpur University, Kolkata DEBASIS MAJI
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
vii
CONTENTS
Title PAGE
ACKNOWLEDGEMTS vi
CONTENTS vii
LIST OF TABLES xi
LIST OF FIGURES xii
ACRONYMS
CHAPTER 1: INTRODUCTION
1.1. Introduction 2
1.2. Motivation 3
1.3. Problem Statement 5
1.4. Objectives 5
1.5. Report Organization 6
CHAPTER 2: LITERATURE REVIEW
2.1. Image Fusion 8
2.2. Image Segmentation 14
CHAPTER 3: DESIGN THE PROPOSED IMAGE FUSION AND
SEGMENTATION MODEL
17
CHAPTER 4: IMAGE FUSION
4.1.Introduction 19
4.2. Pixel level Image Fusion 20
4.3. Feature level Image Fusion 21
4.4.Decision level image fusion 22
4.5.Fusion evaluation method 22
4.6. Summary 23
CHAPTER 5: OVERVIW OF PRINIPAL COMPONENT
ANALYSIS BASED IMAGE FUSION
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
viii
5.1. Introduction 25
5.2. PCA Details 26
5.3. Properties of PCA 28
5.4. Dimensionality Reduction and Feature Extraction 29
5.5. PCA based Image Fusion 30
5.6. PCA Application in Image Fusion 31
5.7. Summary 31
CHAPTER 6: DCT BASED IMAGE FUSION
6.1. DCT Application in Image Fusion 33
6.2. Simulation Results and Discussion 40
6.3. Conclusion 41
CHAPTER 7: IMAGE STRUCTURAL SIMILARITY-BASED
METRICS
7.1. SSIM Review 43
7.2. Summary 45
CHAPTER 8: ARTIFICIAL BEE COLONY OPTIMIZATION 47
CHAPTER 9: IMPLEMENTED MODEL 53
CHAPTER 10: FUSION RESULTSAND CASE STUDIES
10.1. Comparison between DCT & Optimization Fusion Results 55
10.2. Conclusion 61
CHAPTER 11:OVERVIW OF IMAGE SEGMENTATION
11.1. Challenges for Image Segmentation 63
11.2. Overview of Segmentation Algorithms 64
CHAPTER 12: GSA-K MEANS CLUSTERING CHEN AND VESE
SEGMENTATION
12.1. Chan and vese 66
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
ix
12.1.1. Introduction 66
12.1.2. The Chan and Vese Model (Piecewise constant model) for
Image Segmentation
67
12.1.3. Level Set Formulation of the Model 68
12.1.4. The Chan and Vese Algorithm 70
12.1.5. Strengths and Drawbacks of the Chan and Vese Algorithm 71
12.1.6. Extension of Chan Vese algorithm for Vector-valued images 72
12.2. K-means Clustering Algorithm 74
12.3 GSA 74
12.4. Proposed HGSA-k means based Chan and Vese model 75
12.5. Simulation Results and Discussions 77
12.6 Conclusion 82
CHAPTER 13: CONCLUSION
13.1. Conclusion 84
13.2. Future Scope 85
REFERENCES 86
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
x
LIST OF TABLES
Table Title PAGE
5.1 Mutual Information in Fused Image 31
6.1 The Joint-entropy an Mutual Information algorithms, a
measure of quality of the fused image
40
8.1 The basic Artificial Bee Colony Algorithm 49
10.1 Comparing the resultant images of DCTav and DCTopti by
SSIM
56
10.2 Comparing the resultant images of DCTav and DCTopti by
SSIM
58
12.1 K-means clustering Algorithm 74
12.2 GSA K-means approach Algorithm 75
12.3 Timing comparison between the standard C-V model and our
proposed C-V model for segmentation
78
12.4 The segmentation performance of our proposed algorithm 81
LIST OF FIGURES
Figure Title PAGE
3 Proposed Model 17
4.1 Level classification of the various popular image fusion
methods based on computation source
20
4.2.1 A Schematic of Pixel level fusion process 21
4.3.1 Schematic of feature level fusion process 22
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
xi
4.4.1 A Schematic of decision level fusion process 22
5.5.1 PCA based image fusion 30
5.6.1 (a) IR image, (b) Visible image, (c) Fused image 31
6.1.1 Block diagram of DCT based image fusion algorithm 35
6.2.1 Results obtained by image fusion using the DCT algorithms 40
7.1 Diagram of the Structural Similarity (SSIM) measurement
system
43
8.1 The flow chart for the Basic Artificial Bee Colony algorithm 48
9 Implemented Model 53
10.1.1 Results obtained after fusion with different schemes 55
10.1.2 Results obtained after fusion with different schemes 57
10.1.3 Results obtained by image fusion using the proposed
DCTopti algorithm
58
10.1.4 Results obtained by image fusion using the proposed
DCTopti algorithm
59
10.1.5 Results obtained by image fusion using the proposed
DCTopti algorithm
60
12.4.1 Binary tree structure of hierarchical segmentation 76
12.5.1 Sample PCA fused images (a, b, c, d) on which the
experiment has been done
77
12.5.2 Two-class segmentation of the sample images 78
12.5.3 Mean average standard deviation of the GF model 79
12.5.4 Mean average standard deviation value from our proposed
model
80
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
xii
ACRONYMS
SYMBOL DESCRIPTION
DCT Discrete Cosine Transform
IR infrared
ABC Artificial Bee Colony
SSIM Structural Similarity Index Measure
PCA Principal Component Analysis
PSNR Peak Signal –to- Noise Ratio
WT Wavelet Transformation
C-V Chan-Vese
NMSD Non-Multi-scale-Decomposition
SPM Segmentation Performance Measure
PET positron emission tomography
MMW millimeter wave
PSO Particle Swarm Optimization
GA Genetic Algorithm
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
CHAPTER 1:CHAPTER 1:CHAPTER 1:CHAPTER 1:
INTRODUCTION
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 1 Page 2
Chapter 1
Introduction
1.1 INTRODUCTION
pplications of advanced surveillance system using multi-modal imagining sensors for
enhancement of vision systems and its performance under challenging environmental conditions,
obstructed view points, and different type of targets has continually been upgraded in many of
the recent research works. Image fusion techniques are gradually increasing as a potential means
of combining two (or more) images for maximizing information content of the resultant image.
Fusion algorithms are applied in a priori image combination for the purpose of object tracking,
recognition, detection, or classification. To perceive minute details from visible images is
difficult due to the presence of shadows, variations in illumination, reflections, and etc. while; on
the other hand, infrared (IR) imaging is to a certain extent unaffected by most of the
aforementioned factors. Image fusion is a technique which combines two or more images into a
single image called the fused image. The content of information stored in the fused image is
greater than that of the constituent input images. Image fusion has recently been, extensively,
implemented in the fusion of Infrared (IR) and visual images.
In this research work a novel Discrete Cosine Transform (DCT) based patch-wise image
fusion technique, using Artificial Bee Colony (ABC) based optimization, has been formulated to
fuse IR and Visible images. For the sake of comparison of quality of the fused images with
respect to other existing algorithms Structural Similarity Index Measure (SSIM) has been used.
A
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 1 Page 3
The superior result of the proposed algorithm shows that it is highly robust and stable and
provides better results with respect to other existing DCT based algorithms.
1.2 Motivation:
Image fusion techniques usually used for image processing where a single image sensor is not
enough to provide required data. So, fusion is adopted where direct object perception is difficult
and noise prone or expensive.
Two types of images are commonly fused to get grater quality of information.IR sensor will
make complement of image information with visible image range. Visible images offer a rich
content where a detection of people /object can now ever be limited by change in lighting
conditions. IR images generally allow a better contrast which is obtained between a
object/person and the environment, but these images are not as robust to changes in temperature
and wind combination.
So, multi sensor data has become a discipline which demands more general formal solutions to a
number of application cases. Several situations in image processing require both high spatial and
high spectral information in a single image. This is important in remote sensing. However, the
instruments are not capable of providing such information either by design or because of
observational constraints. One possible solution for this is data fusion.
We can get some examples in image fusion in different field of application and need for fusion.
One example is medical image fusion where we have to access organs inside the body, which are
not directly accessible. So few no of sensors are used to get measurements of a different tissue
property. Information from a single sensor is fraught with common problems related to sensor
noise, physical constraints and obstruction of view, shadowing, tissue movement and patient
motion among others. Sensor fusion methods are then adopted to parse through the information
from multiple sensors, perform complex deductions and provide consistent conclusions.
Required margins of error from an automated fusion and inference system are required to be at
least as low as that arrived at by the physician herself. An example of the use of fusion is in
radiotherapy treatment, where CT and MRI are employed to provide complimentary soft-tissue
hard-tissue information in the brain and skull. In these medical applications, the sensor and
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 1 Page 4
sensing technology remains the same but the fusion occurs over data collected at two different
time points.
Second example is artificial intelligence imaging system where multiple environment parameters
are collected by sensors leading to an information overload from the glut of accumulated data.
Automated systems are required to provide reliable decisions in a timely fashion. The amount of
time needed to reach a reliable decision increases rapidly with the amount of information
available. Sensor fusion is necessary to combine information in a way that removes
inconsistencies and presents clearly the best interpretation of measurements input from many
individual sources.
Sensor fusion combines input from many independent sources of limited accuracy and reliability
to give information of known accuracy and proven reliability. Calibration required for accuracy
and reliability. Medical imaging applications use expensive sensors that are calibrated on a
regular basis and are maintained to perform at high standards. Other applications require that
sensors be placed in hostile environments where access is reduced or eliminated, such as in outer
space, deep sea, forests, mountains or river beds. Due to this reason, applications such as
monitoring soil toxicity or water contamination can be addressed by distributing several
hundreds of cheap sensors in the environment.
Lastly, diversity in sensor types allows sensing of a variety of object or environment properties.
Diversity could be achieved by harnessing sensors that focus on different bands in the
electromagnetic spectrum. For example, visible and infrared sensors can be used in security
systems, or visible light and audible sound can be used in a video camera. This diversity leads to
a reduction in the probability of decision error and uncertainty encountered in the measurements
thus making the sensing system more reliable which ultimately benefits the inference-making
procedure.
So as a summary the advantages of sensor fusion over single sensor processing are due to the
redundancy, diversity and complementarily among multiple sensors. When data from multiple
sensors is fused together the resultant observation is expected to have a higher signal to noise
ratio, a reduction in overall measurement variance and a better and more sophisticated picture of
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 1 Page 5
the environment. Redundancy is caused by the use of multiple sensors to measure the same
entity. It is well known that redundancy reduces uncertainty.
1.3 Problem Statement:
From the survey on the image fusion and segmentation it observed that:
The purpose of advanced surveillance system using multi-mode imaging sensors for
enhancement of vision system and its performance under challenging environmental conditions,
and different type of targets has continually been upgraded in many of the resent research works.
The detection of the moving persons has become more and more important over the past few
years. Various applications in the area of security and surveillance are promising. The objective
of the research work demonstrated in this chapter is to develop a new system which combines an
IR and visible sensor to enable the detection and surveillance of pedestrians over a period of
time. More specifically, we will focus the problems in an environment where pedestrians are
moving in a range of specified distances within an area affected by various lighting and
atmospheric conditions.
1.4 Objectives:
(i) Development of a novel Discrete Cosine Transform (DCT) based patch-wise image
fusion technique, using Artificial Bee Colony (ABC) based optimization.
(ii) Development of Cosine Transform (DCT) image fusion technique using Artificial
Bee Colony (ABC) based optimization, of a method of intelligent fusion which will
enable the robustness of human detected to be improved while reducing false alarms
and the advent of non detected pedestrians.
(iii) For the sake of comparison of quality of the fused images with respect to other
existing algorithms Structural Similarity Index Measure (SSIM) has been used.
(iv) Development and implementation of an improved image fusion technique based on
proposed algorithm shows that it is highly robust and stable and provides better
results with respect to other existing DCT based algorithms.
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 1 Page 6
1.5 Report Organization:
The thesis report is organized as follows:
In Chapter 2 a Literature Review is presented. Chapter 3 contains the Designe the
proposed image fusion and Segmentation Model of the thesis work. Chapter 4 & Chapter
5&6 briefly describe the Principal Component Analysis and Proposed DCTopti Image
Fusion algorithm. Chapter 7 a Image Structural Similarity-Based Metrics and Chapter 8
Artificial Bee Colony Optimization are briefly describe mathematical and algorithms..Chapter
9 draws the Implemented Model. Chapter 10 several Case studies are described. GSA-K
Means Clustering Chen and Vese Image Segmentation is presented and discusses result in
Chapter 11 and Chapter 12. Finally Chapter 13 gives a Conclusion & describes the Future
Scopes of the implemented image Fusion and image Segmentation algorithms. The References
are given in end.
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
CHAPTER 2: CHAPTER 2: CHAPTER 2: CHAPTER 2:
LITERATURE REVIEW
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 2 Page 8
Chapter 2
Literature Review
2.1 Image Fusion:
Image Fusion is used extensively in image processing systems. Various Image Fusion methods
have been proposed in the literature to reduce blurring effects. Many of these methods are based
on the post-processing idea. In other words, Image fusion enhances the quality of image by
removing the noise and the blurriness of the image. Image fusion takes place at three different
levels i.e. pixel, feature and decision. Its methods can be broadly classified into two that is
special domain fusion and transform domain fusion. Averaging, Bravery method, Principal
Component Analysis (PCA), Discrete Cosine Transform (DCT) based methods are special
domain methods. But special domain methods produce special distortion in the fused image
.This problem can be solved by transform domain approach. The multi-resolution analysis has
become a very useful tool for analyzing images. Therefore, fusion of IR and Visual images is a
potential solution for improvement in person detection, tracking and recognition even under the
presence of challenging situations like: night-time imaging, foggy weather imaging, etc. A brief
summary of the literature is given below:
Toet et al. [4] proposed a cognitive image fusion scheme wherein a study was conducted to
investigate the qualitative relative difference in human visual perception between the component
input images and the visual images. The method proposed is semi-automatic as human subjects
were asked to derive a reference contour image based on semantically meaningful contiguous
regions in a set of component individual images. The reference contour images were used in a
bid to optimize, using a precision-recall framework, so as to enhance the performance of
different image fusion methodologies.
Wang et al. [20] proposed a robust fusion algorithm of infrared and visible images for detecting
person, tracking, recognition, and fusion performance.
Zin et al. [3] proposed an algorithm on person detection using the resultant image of fusion of
thermal and visual image using multi-slit method and movement of Gravity Center (GC)
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 2 Page 9
patterns. The quality of the fused image can be assessed by using Structural Similarity Index
Measure (SSIM) as proposed by Brooks et al. [6]. In this paper the quality of the fused image
has been determined using the SSIM score.
Extensive use of Principal Component Analysis (PCA) has been found in recent research works,
especially in the field of image fusion. Patil et al. [8] proposed an image fusion strategy using
hierarchical PCA, wherein a study of PCA and pyramid decomposition based image fusion was
performed.
Desale, R.P et al. [33] explained that the Image Fusion is a technique of combining the
appropriate information from a set of images, into a single image, in which the resultant fused
image will be more useful and absolute than any of the input images. This paper discusses the
Formulation, Process Flow Diagrams and algorithms of PCA (principal Component Analysis),
DCT (Discrete Cosine Transform) and DWT based image fusion techniques. The results are also
presented in table & picture format for comparative analysis of above techniques. The DCT &
PCA are conventional fusion techniques with many drawbacks, whereas DWT based techniques
are more favorable as they provides better results for image fusion. In this paper, two algorithms
based on DCTav and DCTopti are proposed.
Prakash, C et al. [34] described that the Image fusion is basically a process where multiple
images (more than one) are combined to form a single resultant fused image. This fused image is
more creative as compared to its original input images. The fusion technique in medical images
is useful for resourceful disease analysis purpose. This paper illustrates different multimodality
medical image fusion techniques and their results assessed with various quantitative metrics.
Firstly two registered images CT (anatomical information) and MRI-T2 (functional information)
are taken as input. Then the fusion techniques are applied onto the input images such as
Mamdani type minimum-sum-mean of maximum(MIN-SUM-MOM) and Redundancy Discrete
Wavelet Transform (RDWT) and the resultant fused image is analyzed with quantitative metrics
namely Over all Cross Entropy(OCE),Peak Signal –to- Noise Ratio (PSNR), Signal to Noise
Ratio(SNR), Structural Similarity Index(SSIM), Mutual Information(MI). From the derived
results it is inferred that Mamdani type MIN-SUM-MOM is more productive than RDWT and
also the proposed fusion techniques provide more information compared to the input images as
justified by all the metrics.
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 2 Page 10
Mohamed, M et al. [35] has define the Image fusion is a process which combines the data from
two or more source images from the same scene to generate one single image containing more
precise details of the scene than any of the source images. Among many image fusion methods
like averaging, principle component analysis and various types of Pyramid Transforms, Discrete
cosine transform, Discrete Wavelet Transform special frequency and ANN and they are the most
common approaches. In this paper multi-focus image is used as a case study. This paper
addresses these issues in image fusion: Fused two images by different techniques which present
in this research, Quality assessment of fused images with above methods, Comparison of
different techniques to determine the best approach and Implement the best technique by using
Field Programmable Gate Arrays (FPGA). First of these techniques is presented and then each
fusion method is performed on various images. In addition experimental results are
quantitatively evaluated by calculation of root mean square error, entropy; mutual information,
standard deviation and peak signal to noise ratio measures for fused images and a comparison is
accomplished between these methods. Then we chose the best techniques to implement them by
DCT algorithms.
Haghighat, M et al.[36] has explained that the image fusion is a technique to combine
information from multiple images of the same scene in order to deliver only the useful
information. The discrete cosine transformation (DCT) based methods of image fusion are more
suitable and time-saving in real time system. In this paper an efficient approach for fusion of
multi-focus images based on variance calculated in DCT domain is presented. The experimental
result shows the efficiency improvement of our method both in quality and complexity reduction
in comparison with several recent proposed techniques.
Pei, Y et al. [37] explained that this paper proposes an improved discrete wavelet framework
based image fusion algorithm, after studying the principles and characteristics of the discrete
wavelet framework. The improvement is the careful consideration of the high frequency subband
image region characteristic. The algorithms can efficiently synthesis the useful information of
the each source image retrieved from the multi sensor. The multi focus image fusion experiment
and medical image fusion experiment can verify that our proposed algorithm has the
effectiveness in the image fusion. On the other side, this paper studies the quality assessment of
the image fusion, and summarize and quantitatively analysis the performance of algorithms
proposed in the paper.
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 2 Page 11
Li, H et al. [38] has discussed that in this paper, the wavelet transforms of the input images are
appropriately combined, and the new image is obtained by taking the inverse wavelet transform
of the fused wavelet coefficients. An area-based maximum selection rule and a consistency
verification step are used for feature selection. A performance measure using specially generated
test images is also suggested.
The PCA image fusion method [47] basically uses the pixel values of all source images at each
pixel location, adds a weight factor to each pixel value, and takes an average of the weighted
pixel values to produce the result for the fused image at the same pixel location. The optimal
weighted factors are determined by the PCA technique. The PCA image fusion method reduces
the redundancy of the image data.
Super-resolution (SR) reconstruction [48] is a branch of image fusion for bandwidth
extrapolation beyond the limits of a traditional electronic image system. Katartzis and Petrou
describe the main principles of SR reconstruction and provide an overview of the most
representative methodologies in the domain. The general strategy that characterizes super-
resolution comprises three major processing steps which are low resolution image acquisition,
image registration/motion compensation, and high resolution image reconstruction. Katartzis
and Petrou presented a promising new approach base on Normalized Convolution and a robust
Bayesian estimation, and perform quantitative and qualitative comparisons using real video
sequences.
Mitianoudis and Stathaki demonstrate the efficiency of a transform constructed using
Independent component Analysis (ICA) and Topographic Independent Component Analysis
based for image fusion in this study [49]. The bases are trained offline using images of similar
context to the observed scene .The images are fused in the transform domain using novel pixel-
based or region-based rules .An unsupervised adaption ICA-based fusion scheme is also
introduced. The proposed schemes feature improved performance when compared to approaches
based on the wavelet transform and a slightly increased computational complexity. The authors
introduced the use of ICA and topographical ICA based for image fusion applications .These
bases seem to construct very efficient tools, which can complement common techniques used in
image fusion, such as the Dual-Tree Wavelet TRANSFORM. The proposed method can
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 2 Page 12
outperform the wavelet approaches. The Topographical ICA based method offers a more
accurate directional, thus capturing the salient features of the image more accurately.
He, D et al. [39] explained that the main objective of image fusion is to create a new image
regrouping the complementary information of the original images. The challenge is thus to fuse
these two types of images by forming new images integrating both the spectral aspects of the low
resolution images and the spatial aspects of the high resolution images. The most commonly
used image fusion techniques are: Principal Components Analysis (PCA), Intensity-Hue-
Saturation Transformation (IHS), High Pass Filter (HPF) and Wavelet Transformation (WT).
The PCA and IHS, are simple to use but they are highly criticized because the resulting image
does not preserve faithfully the colors found in the original images. The HPF method is sensitive
to the filtering used (filtering type, filter window size, etc.) and the mathematical operations
used.
Y-T, K et al. [40] has discussed in this paper the Histogram equalization is widely used for
contrast enhancement in a variety of applications due to its simple function and effectiveness.
Examples include medical image processing and radar signal processing. One drawback of the
histogram equalization can be found on the fact that the brightness of an image can be changed
after the histogram equalization, which is mainly due to the flattening property of the histogram
equalization.
T.Zaveri, M et al. [41] explained that the Image fusion is a process of combining multiple input
images of the same scene into a single fused image, which preserves relevant information and
also retains the important features from each of the original images and makes it more suitable
for human and machine perception. In this paper, a novel region based image fusion method is
proposed. In literature shows that region based image fusion algorithm performs better than pixel
based fusion method. Proposed algorithm is applied on large number of registered images and
results are compared using standard reference and no reference based fusion parameters.
O, R et al. [42] has discussed a novel approach for the fusion of spatially registered images and
image sequences. The fusion method incorporates a shift invariant extension of the discrete
wavelet transform, which yields an over complete signal representation. The advantage of the
proposed method is the improved temporal stability and consistency of the fused sequence
compared to other existing fusion methods. We further introduce information theoretic quality
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 2 Page 13
measure based on mutual information to quantify the stability and consistency of the fused image
sequence.
Ghimire, D et al. [43] has discussed that the main objective of image enhancement is to improve
some characteristic of an image to make it visually better one. This paper proposes a method for
enhancing the color images based on nonlinear transfer function and pixel neighborhood by
preserving details. In the proposed method, the image enhancement is applied only on the V
(luminance value) component of the HSV color image and H and S component are kept
unchanged to prevent the degradation of color balance between HSV components. The V
channel is enhanced in two steps. First the V component image is divided into smaller
overlapping blocks and for each pixel inside the block the luminance enhancement is carried out
using nonlinear transfer function. In the second step, each pixel is further enhanced for the
adjustment of the image contrast depending upon the center pixel value and its neighborhood
pixel values. Finally, original H and S component image and enhanced V component image are
converted back to RGB image.
Sruthy, S et al. [44] has focused on the development of an image fusion method using Dual Tree
Complex Wavelet Transform. The results show the proposed algorithm has a better visual quality
than the base methods. Also the quality of the fused image has been evaluated using a set of
quality metrics.
Patil, U et al. [45] has focused on image fusion algorithm using hierarchical PCA. Authors
described that the Image fusion is a process of combining two or more images (which are
registered) of the same scene to get the more informative image. Hierarchical multi scale and
Multiresolution image processing techniques, pyramid decomposition are the basis for the
majority of image fusion algorithms. Principal Component analysis (PCA) is a well-known
scheme for feature extraction and dimension reduction and is used for image fusion. We propose
image fusion algorithm by combining pyramid and PCA techniques and carryout the quality
analysis of proposed fusion algorithm without reference image.
Aribi, W et al. [46] explained that the quality of the medical image can be evaluated by several
subjective techniques. However the objective technical assessments of the quality of medical
imaging have been recently proposed. The fusion of information from different imaging
modalities allows a more accurate analysis. We have developed new techniques based on DCT
opti fusion. MRI and PET images have been fused with eight multi resolution techniques. The
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 2 Page 14
results proved that DCTav and DCTopti techniques to offer the best results.
2.2 Image Segmentation:
Image segmentation, i.e. partitioning an image into homogeneous areas, is one of the most
fundamental problems in a variety of applications, including but not limited to remote sensing,
optical imaging, and medical image analysis. It have been made toward a general segmentation
scheme, many difficult challenges still exist for many problems, especially on medical images.
For example, poor image contrast and noise are very common for many modalities, such as
ultrasound, positron emission tomography (PET) .Chan-Vese (C-V) model is a very standard
active contour based approach in image segmentation. This model uses region-based information
in its level set based formulation and tries to minimize an energy fitting functional associated
with it by solving an Partial Differential Equation. Though this model can segment internal
objects very well, it still suffers from the problem of getting stuck at local minimum due to the
fitness functional being a non-convex and non-unique one.
In this work, a method to predetermine the mean value of intensity within and outside the active
contour is proposed which significantly improves the computation time of the C-V model. In
addition the model is less sensitive to contour initializations and is found to converge to the
global minimum in almost all cases. A brief summary of the literature is given below:
A unique method developed by Gibou and Fedkiw [50] was developed that solves the C-V
model by using the k_means clustering algorithm. Although this method [51] was found to be
much faster than the standard C-V model and did not require the need for level sets, it still
suffers from the problem of getting stuck at local minimum if the initial cluster points are not
appropriately selected. This error increases as the number of classes into which the image has to
be segmented increases.
In this work, evolutionary algorithms have been used in conjunction with the k_means algorithm
to take care of this problem. Successful implementations have been done using the Gravitational
Search Algorithm (GSA) [52] to segment images into two-class, three-class and four-class
effectively. Our model is seen to outperform the model suggested by Gibou and Fedkiw[50].
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 2 Page 15
Active region based image segmentation is generally carried out by Chan-Vese (C-V) model [53]
by wrapping around [53][54], an initial active contour [55] along the steepest descent direction
of energy employing gradient descent search (GDS) method [56]. The Chan-Vese model for
image segmentation is well developed and well-cited enough. The formulation of Chan-Vese
model usually comes up with the requirement of solving the partial differential equation (PDE)
for obtaining the route of contours evolved during the process of computation employing level
set formulation [53][54].Consequently, the associated energy gradients are derived by using the
Euler-Lagrange equations. In this respect, the GDS method is a convenient tool as they can be
utilized for minimization of non-convex functional and easy to implement as they involve
calculation of first order derivatives. But it generally converges to the first local minimum it
encounters and its rate of convergence is often very slow. For the sake of brevity, in this work we
presume the findings of the related previous works to be truthful i.e. the C-V energy fitting
functional [53] is non-convex and non-unique in nature and may also have many local minima.
Thus to alleviate the
problem of getting stuck to local minima, this work proposes a modified gradient search
technique that guarantees better and faster convergence of the C-V algorithm towards its global
minimum to get accurate segmentation results compared to the wellestablished heuristic
searches. The present work mainly rests on the efficacy of a variant of GDS, namely, the Delta-
Bar-Delta rule (DBR), proposed by Jacobs [57],which utilizes a learning parameter update rule,
in each iteration, in addition to the weight update rule. This method bears similarity with the
RPROP [56] method, although the update rules are quite distinct in nature. In this work, we
propose a modified version of DBR algorithm, namely MDBR algorithm, which utilizes a
modified version of DBR algorithm to update learning rate parameters and momentum method to
update weights, to achieve even faster convergence. The proposed Chan-Vese-MDBR algorithm
has been utilized to segment both scalar and vector valued images and its superiority has been
firmly established in comparison with other popular search methods used for level set based
image segmentation algorithms e.g. the well established basic GDS model and recently proposed
momentum (MOMENTUM),resilient backpropagation (RPROP) and conjugate gradient
(CONJUGATE) based learning methods.
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
CHAPTER 3:CHAPTER 3:CHAPTER 3:CHAPTER 3:
Design the proposed image fusion and
segmentation Model
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 3 Page 17
Chapter 3
Design the proposed image fusion and
Segmentation Model
3. Proposed image fusion and Segmentation Model:
Proposed image fusion and Segmentation Model
Pre - Processing
Image Fusion Domain
Developed a new
Fusion algorithm
Post - Processing
Segmentation using
Modified or developed
Chan – Vese
Segmentation
Image Enhancement
Or fused image quality
measurement
Performance Evaluation
Segmentation
Result Display
Fusion
Result Display
Raw image
Infrared
image
Visual image
Back
Ground
subtraction
Object detection
Analysis
Fig.3. Proposed Model
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
CHAPTER 4CHAPTER 4CHAPTER 4CHAPTER 4::::
IMAGE FUSION
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 4 Page 19
Chapter 4
Image Fusion
4.1. Introduction:
The fast modern development of the technique of sensors, micro-electronics, and multi-
sensor systems, fusion algorithms appreciably reduce the amount of raw data that needs to be
presented or processed without loss of information content as well as provide an effective way of
information integration. Day by day numerous image fusion algorithms are developed to address
the growing need for image fusion. The algorithms developed into two groups; Multi-scale-
Decomposition (MSD)-based fusion methods, and Non-Multi-scale-Decomposition (NMSD)-
based fusion methods [3]. All NMSD are not based on multi-scale transforms. Most common
NMSD fusion methods includes, Principal Component Analysis (PCA), DCT.
Image fusion techniques can also be classified based on the level of Processing where the fusion
takes place (Hall, 2001) [3]. There are three main levels where image fusion may take place and
they include.
Pixel Level
Feature Level and
Decision Level.
Level classification of the various popular image fusion methods based on computation source
that illustrates them is shown in Fig.3.1.
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 4 Page 20
Image Fusion
Pixel-level image
fusion
Averaging
DCT
PCA
WaveletTransform
Intensity Hue Saturation (IHS) Transform
Feature-level image fusion Decision-level image fusion
Neural Networks
Region-BasedSegmentation
K-means Clustering
Similarity Matching to Content-Based image Retrieval
Fusion Based on Fuzzy andUnsupervised FCM
Fusion Based on SupportVector Machine
Fusion Based on Information Level in the Regions of Images
Fig4.1. Level classification of the various popular image fusion methods based on computation
source
4.2. Pixel level image fusion:
Pixel level image fusion is the lowest processing level referring to the merging of the physical
parameters of the source images. Figure 4.2.1 illustrates schematic of pixel level fusion process.
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 4 Page 21
Infrared image
Visible image
Pixel Fusion Validation Fused image
Fig.4.2.1. A Schematic of Pixel level fusion process [3].
To perform Pixel level fusion all input images need to be spatially registered exactly to all other
input images, so that all Pixel positions of all the input images correspond to the same location in
the real world.
4.3. Feature level image fusion:
Feature level methods are the next stage of processing where image fusion may take place. It
have required to extraction of object (features) from the input images. Since, one of the
important goals of fusion is to conserve the image features, feature level, feature level methods
have the ability to capitulate subjectively better fused images than pixel based techniques. Figure
4.3.1.illustrates a schematic of feature level fusion process.
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 4 Page 22
Fig.4.3.1. Schematic of feature level fusion process [3].
4.4. Decision level image fusion:
Decision level techniques are the highest level of processing where image fusion can take
place. Fusion at the Decision level takes Feature level fusion one step further by declaring
identities to the objects recognized, by the individual input images, and then assigning a quality
measure to the extracted features-see figure 4.4.1.
Infrared image
Visible image
Feature
extraction
Feature
extraction
object recognition
Decision level
fusion
object recognition
Results
Fig.4.4.1. A Schematic of decision level fusion process [3].
4.5. Fusion evaluation methods:
The target of image fusion is to create a realistic and combined image that retains the
important information from the source images while minimizing the noise caused by fusing the
images. For the application, these images will be typically viewed and interpreted by an operator.
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 4 Page 23
A number of evaluation approaches and metrics have been proposed to quantify image fusion
performance.
4.6. Summary:
For image fusion research is mainly due to the contemporary developments in the fields of
multi-spectral, high resolution and cost effective image sensor design technology. Pixel-level
image fusion algorithms represent an efficient solution of operator related information overload.
Fusion effectively reduces the amount of data that needs to be processed without any significant
loss of useful information and also integrates information from multi-spectral sensors.
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
CHAPTER 5CHAPTER 5CHAPTER 5CHAPTER 5::::
OVERVIEW OF PRINCIPAL COMPONENT
ANALYSIS BASED IMAGE FUSION
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 5 Page 25
Chapter 5
Overview of Principal Component Analysis
5.1. Introduction:
Sometimes, the original data representation will be redundant for some reasons, i.e. some
variables will have a variation smaller than the measurement noise and thus will be irrelevant,
sometimes the original data is too big that it cannot be expressed because of lack of time.
Principal Component Analysis is a way to reduce features to some extent. It was introduced by
Karl Pearson in the year of 1901 [79] is a powerful process for extracting structure from possibly
high dimensional data sets. It is a statistical technique which uses orthogonal transformation to
convert a set of observations of possibly correlated variables called principal component. The
number of original variables is greater than or equal to the number of principal components. The
analysis is such a way that the first principal component always has the largest possible variance
and the second principal component always having the second largest possible variance, and so
on. If the data set is jointly normally distributed, it is guaranteed that principal components
should be independent. Depending on the field of signal processing application, it is also called
the discrete Karhunen-Loeve transform (KLT). It is readily performed by solving an Eigen value
problem or by using iterative algorithms which estimate principal components. PCA can produce
with lower-dimensional picture, a projection of this object when viewed from its most
informative point of view.
5.2. PCA Details:
PCA finds the linear projection of high dimensional data into a lower dimensional
subspace such as:
1. The variance retained is maximized.
2. The least square reconstruction error is minimized.
It is an orthogonal transformation of the coordinate system in which we describe our data. This
technique transforms the data to a new coordinate system such that the greatest variance by some
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 5 Page 26
projection of the data comes to lie on the first coordinate, which is called first principal
component and the second greatest variance lies on the second coordinate, which is called
second principal component and so on.
Mathematically this transform can be defined as a set of p-dimensional vectors of weights
)(21)( ),.......,,( kpk wwwW = that map each row vector )(iX of X to a new vector of principal
component scores ,),.....,( )(21)( ipi tttt = given by )()()( . kiik WXt = in such a way that all the
individual variables of t considered over the data set successfully inherit the maximum possible
variance from x, with each leading vector W constrained to be a unit vector.
First Component:
The first principal component )1(W can be stated as:
2
)(
2
)(1)1( ).(1
maxarg)(
1
maxarg∑∑
==
==
i
i
i
i WXW
tW
W
Equivalently, the equation can be written in matrix form as:
1
maxarg
1
maxarg 2
)1( XWXWW
XWW
W TT
==
==
Since )1(W has been defined to be a unit vector,
maxarg)1(WW
XWXWW
T
TT
=
)1(W Found, the first component of a data vector )(lX can then be given as a score )1()()(1 .WXt ll = in the
transformed coordinates, or as the corresponding vector in the original variables, .. )1()1()( WWX l
Further components:
The thK component can be found by discarding the first 1−k principal components from X:
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 5 Page 27
T
S
k
S
Sk WXWXX )(
1
1
)(1ˆ ∑
−
=
−−=
Loading vector,
ˆˆ
1
maxargˆ
1
maxarg11
2
1)(WW
WXXW
WWX
WW
T
k
T
k
T
kk
−−
−=
==
=
The full principal components decomposition of X can therefore be given as:
XWT =
Where W is a p-by-p matrix whose columns are the eigenvectors of .XX T
Covariance:
Empirical sample covariance matrix of dataset X is proportional to XX T
The sample covariance Q between two of the differential principal components over the dataset
can be given as:
)).((),( )()()()( kjkj XWXWPCPCQ α
)()( k
TT
j XWXW=
)()()( kk
T
j WW λ=
)()()( k
T
jk WWλ=
Where the Eigen value property of )(kW has been used to move from line 2 to line 3.
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 5 Page 28
Other way to characterize the principal components transformation is therefore as the
transformation to coordinates which diagonals the empirical sample covariance matrix as:
TT WWXXQ Λ=α
Λ=Λ WWWWQWW TTT α
Where, Λ is the diagonal matrix of Eigen values )(kλ of XX T
5.3. Properties of PCA:
Property 1: qp ≤≤1 , where p is an integer, the orthogonal linear transformation xBy `= where y
is the p-element vector and B` is a (qxq) matrix, assume that, ∑∑ = BBy ` be the variance
covariance matrix for y. Where, B` is the transposition of B. Trace of ∑y
denoted by ∑
ytr )(
maximized by taking pAB = , here pA consists of the first p columns of A.
Property 2: For that orthogonal transformation xBy `= , with x, B, A and ∑y
defined as earlier,
then ∑
ytr )( can considered to be minimum by taking *
pAB = where *pA consists of the last p
columns of A.
The mathematical implication of this second property is that the last few PCs are not
simply unstructured left-over after removing the important PCs. Because these last PCs have
variances as small as possible they are useful in their own right. They can help to detect
unsuspected near-constant linear relationships between the elements of x, and they may also be
useful in regression, in selecting a subset of variables from x, and in outlier detection.
Property 3: ''222
'111 ..... qqq ααλααλααλ +++=∑
Diagonal elements, ∑
=
=Q
kkjkjx
1
2)var( αλ
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 5 Page 29
Then, perhaps the implication of the result is that not only can we decompose the combined
variances of all the elements of x into decreasing contributions due to each PC, but we can also
decompose the whole covariance matrix into contributions 'kkk ααλ from each PC. Although not
strictly decreasing, the elements of 'kkk ααλ will tend to become smaller as k increases,
as 'kkk ααλ decreases for increasing k, whereas the elements of kα tend to stay 'about the same
size because of the normalization constraints: qkkk ,....,2,1,1'
==αα
5.4. Dimensionality Reduction and Feature Extraction:
Principal Component Analysis allows the extraction of a number of principal components
which can exceeds the input dimensionality. Suppose that the number of observations M exceeds
the input dimension N. PCA even when it is based on the M x M dot product matrix, can find at
most N nonzero Eigen values. They are identical to the nonzero Eigen values of the N x N
covariance matrix.
In this method, computation of the largest Eigen value and the corresponding rescaled
eigenvectors corresponding with the principal components in the feature space. After this we are
getting a column matrix of Eigen values and a matrix of eigenvectors according to the dimension
of the given matrix.
Suppose a matrix having the dimension 200 x 250 has been given as a input to PCA.
Then output of two matrix is obtained. In that one is containing column data of Eigen values
(200 x 1) and another one is having same dimension of input matrix containing rescaled Eigen
vectors (200 x 250) corresponding with the principal components in feature space.
These column data of Eigen values in descending order has been given as a input feature vectors
for classification task. PCA Based Image Fusion.
5.5. PCA Based Image Fusion:
The basic concept of PCA is to transform number of uncorrelated variables (which are
called Principal Components) from number of correlated variables [77].
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 5 Page 30
By Solving Eigenvalue problem, PCA is performed. This is an orthogonal linear
transformation, which transforms data to a new coordinate system. The greatest variance
occupies the first coordinate. In the second coordinate, the second greatest variance lies
[78]. First coordinate is termed as first principal component and so on. Covariance matrix
(C) of data ( tD ) is diagonalizable and defined as: [79]
C = ∑=mi
TiDiD
m1
1 (5.5.1)
Where, tDn
ℜ∈ t = [1, 2, m] and ∑ =mi tD1 = 0.
To spot on the features and reduce the noise, SVD based PCA fusion algorithm is applied
both on VI and IR images.
Figure 5.5.1: PCA based image fusion
5.6. PCA Application in Image Fusion:
Figure 5.5.1 demonstrates the fusing algorithm using PCA. Figure 5.6. 2 show that VI and
IR image is fused and it can be shown that the fused image carries maximum information
than these two inputs [80]. Table 1 shows about the mutual information between fused
image and input images.
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 5 Page 31
(a) (b) (c)
Figure 5.6.1: (a) IR image, (b) Visible image, (c) Fused image
TABLE5.1. Mutual Information in Fused Image
Mutual Information between Visible image and
Fused image (100X100)
100%
Mutual Information between IR image and Fused image
(100X100)
82.85%
5.7. Summary:
The PCA image fusion method simply uses the pixel values of source images at each pixel
location. The PCA technique is useful for image encoding, image data compression, image
enhancement, pattern recognition and image fusion.IR and visible image fusion result compare
with Mutual Information 82.85%.
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
CHAPTER 6CHAPTER 6CHAPTER 6CHAPTER 6: : : :
DISCRETE COSINE TRANSFORMATION
BASED IMAGE FUSION
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 6 Page 33
Chapter 6
Discrete Cosine Transformation Based Image Fusion
6.1. DCT Application in Image Fusion:
DCT has recently, found extensive use in the field of digital image processing. One of the
significant applications of DCT in that field is image fusion. A large number of DCT coefficients
are known to be concentrated in the low frequency region resulting in effective energy
compactness properties [9] [10]. The 2D Discrete Cosine Transform ),( 21 ffX of an image or
2D signal 21)2
,1
(NN
nnX×
ℜ∈ is defined using the following expression (6.1.1).
)2
()1
()2
,1
( ffffX φφ=
)
22
2)1
22(
cos()
12
1)1
12(
cos(
11
01
12
02
)2
,1
(N
fn
N
fnN
n
N
nnnx
++∑−
=∑−
=
ππ (6.1.1)
∀1
220
111
0
−≤≤
−≤≤
Nf
Nf
≤≤
=
=
111,
1
2
01
if,
1
1
)1
(,where
NfN
fN
fφ
−≤≤
=
=
122
1,
2
2
02
if ,
2
1
)1
(
NfN
fN
fφ
21 & ff discrete frequency variables ),( 21 nn pixel index.
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 6 Page 34
Similarly,2D inverse DCT is defined by the following mathematical expression (6.1.2).
( )2
()1
)2
,1
( ffnnx φφ=
)
22
2)1
22(
cos()
12
1)1
12(
cos(
11
01
12
02
)2
,1
()2
()1
(N
fn
N
fnN
n
N
nffXff
++∑−
=∑−
=
ππφφ (6.1.2)
111
0, −≤≤ Nnwhere
122
0 −≤≤ Nn
Image fusion have different categories [19], i.e. (a) Multiview fusion (b) Multimodal fusion (c)
Multitemporal fusion (d) Multifocus fusion (e) Fusion for image restoration. In this section , we
describe six types of DCT image fusion techniques. Fused images are divided into non-
overlaping blocks of size N×N. Here we computed for dct coe fficients rule are implment to get
fused DCT coefficients. The fused image is produced by IDCT fused coefficients. This process is
shown in the figure.6.1.1[2].
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 6 Page 35
Fig.6.1.1. Block diagram of DCT based image fusion algorithm
Therefore, let DCT coefficient matrix is 1X from image patch
1l of image 1 and DCT coefficient
is 2X from image patch
1l of image 2. Suppose cX be the fused DCT coefficients and size of
image patch is pNpN × [2]. The different image fusion techniques proposed in [2] are
discussed as follows:-
a) DCTav:
In this technique each and every one DCT coefficients from mutual image blocks of the
two component images are averaged to get DCT coefficients corresponding to the fused
image. Finally, the inverse DCT of the coefficient matrix gives us the fused image. cX is
defined using the following expression (6.1.3).
)]2
,1
(1
)2
,1
(1
[5.0)2
,1
( ffXffXffcX += (6.1.3)
where, 1,......2,1,02,1 −= pNff
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 6 Page 36
b) DCTma:
All the DC elements corresponding to the two image patches of the component images
are averaged. The AC coefficients with the large magnitude are chosen since these are
those coefficients that correspond to the brightness changes, edges and object boundaries
in the component images. The technique is expressed mathematically by (6.1.4) and
(6.1.5). The aforementioned components do tend to fluctuate in and around zero. Finally,
the inverse DCT of the coefficient matrix gives us the fused image.
)]0,0(1
)0,0(1
[5.0)0,0( XXcX += (6.1.4)
1 ,.......,2,12
,1
,
)2
,1
(2
)2
,1
(1
,),2
,1
(2
)2
,1
(2
)2
,1
(1
,),2
,1
(1
)2
,1
(
−=
⟨
≥=
pNffwhere
ffXffXwhereffX
ffXffXwhereffXffcX
(6.1.5)
c) DCTah:
In this fusion rule, DC coefficients and AC components with the small magnitude are
averaged and the remaining high magnitude AC components are chosen based on their
magnitude, as given by (6.1.6) and (6.1.7).
15.0,.....,1,02
,1
,
)]2
,1
(2
)2
,1
(1
[*5.0)2
,1
(
−=
+=
pNffwhere
ffXffXffcX (6.1.6)
1,.......,25.0,15.0,5.0,,
)2
,1
(2
)2
,1
(1
,),2
,1
(2
)2
,1
(2
)2
,1
(1
,),2
,1
(1
)2
,1
(
21 −++=
⟨
≥=
NNNffwhere
ffXffXwhereffX
ffXffXwhereffXffcX
(6.1.7)
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 6 Page 37
d) DCTcm:
The DC coefficients of the two image patches corresponding to the two component images
are averaged and the AC coefficients are selected based on the larger contrast measure, as
given by mathematical expression (6.1.8) and (6.1.9).
)0,0()0,0(5.0)0,0( 21 XXX c += (6.1.8)
measurecontrast ,
1,.....,2,1,,
),(),(,),,(
),(),(,),,(),(
21
21
212211212
212211211
21
=
−=
⟨
≥=
ηη
ηη
ηη
Nffwhere
ffffwhereffX
ffffwhereffXffX c
(6.1.9)
e) DCTch:
Hence, DCTch fusion is very much similar to DCTah. The lowest magnitude AC
components along with the DC components are averaged and the remaining coefficents
are chosen based on largest contrast measures, as given by expressions (6.1.10) and
(6.1.11).
15.0,.....,2,1,02
,1
,
)]2
,1
(2
)2
,1
(1
[5.0)2
,1
(
−=
+=
Nffwhere
ffXffXffcX (6.1.10)
⟨
≥=
)2
,1
(2
)2
,1
(1
,),2
,1
(2
)2
,1
(2
)2
,1
(1
,),2
,1
(1
)2
,1
(ffffwhereffX
ffffwhereffXffcX ηη
ηη (6.1.11)
1,.....,25.0,15.0,5.02
,1
, −++= NNNffwhere
f) DCTe:
This technique is similar to DCTcm. It has been shown DC components are averaged
together. Those AC components that correspond to the largest energy in the frequency
band are selected, as given by mathematical expression (6.1.12) and (6.1.13).
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 6 Page 38
)]0,0(2
)0,0(1
[5.0)0,0( XXcX += (6.1.12)
≥
≥=
21,),
2,1
(2
21,),
2,1
(1
)2
,1
(
jjwhereffX
jjwhereffX
ffcX ψψ
ψψ (6.1.13)
band
spectralthj aover amplitude avarage2
,1
21
1,.....,2,12
,1
,
=
+=
−=
jj
ffj
Nffwhere
ψψ
DCTopti:
However, we have proposed an image fusion algorithm in this paper that addresses the
problem of weighted combination as a metaheuristic optimization problem as represented
by the following mathematical expressions (6.1.14) and (6.1.15).
)]2
,1
(22
ˆ)2
,1
(11
ˆ[)2
,1
( ffXffXffcX αα += (6.1.14)
where, f
gmin
2,1
arg]2ˆ,
1ˆ[
αααα = (6.1.15)
∑−
=∑−
=−
=1
01
1
02
2)]2,1(2)2,1(1[ )2/1(
2))2,1(1max(10log10
pN
f
pN
f
ffXffXpN
ffXg
∑−
=∑−
=+∑
−
=∑−
=
∑−
=∑−
=−
=1
01
1
02
2)]2
,1
(2
[
1
01
1
02
2)]2
,1
(1
[
1
01
1
02
)]2
,1
(2
)2
,1
(1
[
pN
f
pN
fffX
pN
f
pN
fffX
pN
f
pN
fffXffX
f
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 6 Page 39
The cost function (6.1.15) has been developed as a hybrid formulation of the Peak Signal-to-
Noise Ratio and the Correlation coefficient. The optimization or global minimization of the
objective function given by expression (6.1.15) has been done by the well-known Artificial Bee
Colony optimization which is a recent swarm optimization algorithm for non-convex problems
like that of the formulation of the cost function in our case.
It is most of the spatial domain image fusion methods are complex and time consuming which
are hard to be performed on real-time applications. Moreover, when the source images are coded
in Joint Photographic Experts Group (JPEG) standard or when the fused image will be saved or
transmitted in JPEG format, the fusion approaches which are applied in DCT domain will be
very efficient. To perform the JPEG coding, an image (in color or grey scales) is first subdivided
into blocks of 8x8 pixels. The Discrete Cosine Transform (DCT) is then performed on each
block. This generates 64 coefficients which are then quantized to reduce their magnitude.
The coefficients are then reordered into a one-dimensional array in a zigzag manner before
further entropy encoding. The compression is achieved in two stages; the first is during
quantization and the second during the entropy coding process. JPEG decoding is the reverse
process of coding. We denote A and B as the output images of two cameras that have been
compressed in JPEG coding standard in the sensor agent and further transmitted to fusion agent
of VSN. In the case of using spatial domain method these images must be decoded and
transferred to spatial domain. Then after applying fusion procedure, the fused image must be
coded again in order to be stored or transmitted to an upper node. Tang[36] has considered the
above mention issue of complexity reduction and proposed two image fusion techniques in DCT
domain, namely, DCT + Average and DCT+ Contrast. DCT +Average is calculated by simply
taking the average of all the DCT coefficients of all the input images. This simple method of
averaging leads to undesirable side effects including blurring.
In order to reduce the complication for the real-time applications and also enhance the quality of
the output image, an image fusion technique in DCT domain. Here, the variance of 8×8 blocks
calculated from DCT coefficients is used as a contrast criterion for the activity measure. Then, a
consistency verification (CV) stage increases the quality of output image. Simulation results and
comparisons show the considerable improvement in the quality of the output image and
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 6 Page 40
reduction of computation complexity.
6.2. Simulation result and Discussion:
We test our DCT algorithms on Thermal and visual images as shown in the following fig. 6.2.1
and the resultant fused image obtained is quite satisfactory.
Fig. 6.2.1.Results obtained by image fusion using the DCT algorithms
However, on comparing a thermal image and its corresponding visual images of DCT by the
Joint-entropy an Mutual Information algorithms, a measure of quality of the fused image. We
obtain the following results, as given in table results TABLE 6.1.
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 6 Page 41
TABLE6.1.
Type of
image fusion
Joint-entropy
(Infrared
image)
Joint-entropy
(Visual
image)
Mutual
Information(Infrared
image)
Mutual
Information(Visual
image)
DCTav 8.2756 8.3105 1.187 1.1522
DCTma 8.6732 8.8402 1.4476 1.2807
DCTah 8.4214 8.4656 1.1517 1.1075
DCTe 8.6785 8.8029 1.4796 1.3551
DCTch 8.4208 8.4680 1.1516 1.1044
DCTcm 8.6507 8.8800 1.4772 1.2479
DCTopti 8.2597 8.3956 1.2114 1.0756
6.3 Conclusion:
Six different types of image fusion algorithms based on discrete cosine transform (DCT) and
DCTopti was developed and fused image quality was evaluated using Joint-entropy and Mutual
Information algorithms. DCTopti based image fusion algorithms performed well and these
algorithms are very suitable for real time applications.
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
CHAPTER 7CHAPTER 7CHAPTER 7CHAPTER 7::::
IMAGE STRUCTURAL SIMILARITY INDEX
MEASURE
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 7 Page 43
Chapter 7
Image Structural Similarity Index Measure
7.1. Image Structural Similarity Index Measure Review:
An effective process to judge the quality of the fused image is measuring the loss of structural
information. The image quality measurement, as proposed by Wang et al. [1], is based on
Structural Similarity Index Measure (SSIM). Hence, the structural philosophy can be proposed
for set of equation defining the structural similarity (SSIM) quality metric in image space [6].
For the comparison of two images X and Y, luminance is calculated as the mean of each image
[5] [1].
Fig.7.1. Diagram of the Structural Similarity (SSIM) measurement system [5].
The system diagram of the proposed quality assessment system is shown in fig.7.1. Here, x and y
are two nonnegative image signals, which have been aligned with each other. If we consider one
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 7 Page 44
of the signals to have perfect quality, then the similarity measure can serve as quantitative
measurement of the quality of the second signal.
Two images (or image patches) X and Y to be compared, luminance is estimated as the mean
intensity ( xµ ) , given by equitation (7.1).
∑=
=N
nnx
Nx
1
1µ (7.1)
Standard deviation ( xσ ) is estimated for contrast as given by expression (7.2).
∑=
−−
=N
nxnx
Nx
1
2)(1
1µσ (7.2)
Now, the estimated structure for the image vector X by removing the mean and normalized by
the stander deviation, given by (7.3).
x
xxx
σ
µς
−= (7.3)
Hence, we have used a structural similarity index measure(SSIM) for images added using a
luminance comparison function ),( YXl , the contrast comparison function ),( YXc and structure
comparison function ),( YXs to get a composite measurement using the following expression
(7.4).
γβα )],([)],([)],([),( YXsYXcYXlYXSSIM = (7.4)
where α, β and γ positive constants are used to adjust of three components.
and, the component functions are given by
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 7 Page 45
122
12
),(
kyx
kyxYXl
++
+=
µµ
µµ (7.5)
222
22
),(
kyx
kyxYXc
++
+=
σσ
σσ (7.6)
3
3
3
3,
),(kyx
kxy
kyx
kyxYXs
+
+=
+
+⟩⟨=
σσ
σ
σσ
ςς (7.7)
where <X,Y> is the correlation between the structure of two images.
Now, 2
,,1 23
kkand === βα to get the following mathematical expression(7.8).
)2
22)(1
22(
)2
2)(1
2(),(
kyx
kyx
kyxkyxyxSSIM
++++
++=
σσµµ
σµµ (7.8)
Once the universal image quality index (UIQI) by the following mathematical expression and is
written as [31].
)22
)(22
(
4
22
2.
22
2.),(
yxyx
yxxy
yx
yx
yx
yx
yx
xyyxQ
µµσσ
µµσ
σσ
σσ
µµ
µµ
σσ
σ
++=
++= (7.9)
6.2. Summary:
We have summarized the traditional approach to image quality assessment based on error-
sensitivity, and have enumerated its limitation. We use of structural similarity as an alternative
motivating principle for the design of image quality measures. However, the effectiveness of
these models degrades significantly when applied to a IR and VI images from including a variety
of different types of distortions.
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
CHAPTER 8CHAPTER 8CHAPTER 8CHAPTER 8: : : :
ARTIFICIAL BEE COLONY OPTIMIZATION
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 8 Page 47
Chapter 8
Artificial Bee Colony Optimization
8. Artificial Bee Colony Optimization:
The basic Artificial Bee Colony (ABC) algorithm, for global optimization of high dimensional
numerical problems, was proposed by Karaboga [22]. In comparison to Genetic Algorithm (GA)
and Particle Swarm Optimization (PSO), ABC is characterized with lower computational
complexity, and easier programming [23-24]. The ABC algorithm and its variants have been
used for optimization in a wide spectrum of real world problems like image clustering [25], edge
enhancement [26], multi-level threshold image segmentation [27], training neural networks [28],
path planning of free-flying space robot [29], harmonic estimation [30], and a multitude of other
problems.
The collective intelligent behavior of insect or animal group in nature such as flocks of birds,
Colonies of ants, schools of fish, swarms of bees and termites have attracted the attention of
researchers. The aggregate behavior of insects or animals is called swarm behavior.
Entomologists have studied this collective behavior to model biological swarms, and engineers
applied these models as a framework for solving complex real-world problems [22].
Bee swarms exhibit many intelligent behaviors in their tasks such as nest site building, marriage.
foraging, navigation and task selection. There is an efficient task selection mechanism in a bee
swarm that can be adaptively changed by the state of the hive and the environment. Swarm
intelligence is a research field that models the collective intelligence in swarms of insects or
animals [24].
The flow chart for the basic Artificial Bee Colony (ABC) algorithm is shown in the following
figure8.1.
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 8 Page 48
Fig. 8.1. The flow chart for the Basic Artificial Bee Colony algorithm
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 8 Page 49
The swarm of bee consists of three essential components, food sources, employed foragers, and
unemployed foragers. There are three types of bees in the swarm: employed bees, unemployed
bees, and scouts. There is one employed bee for every food source. The employed bee whose
food source is exhausted by the employed and onlooker bees becomes a scout. The pseuocode
for the ABC algorithm is given as follows.
TABLE 8.1
The basic Artificial Bee Colony Algorithm
BEGIN
- Initialize ABC control parameters.
i) Maximum Cycle Number, MCN , for which the ABC algorithm is iterated.
ii) Number of Employed Bees = Number of Onlooker Bees = SN .
iii) The number of trials, limit , to improve a food source after which it is abandoned.
iv) The dimension of each food source, Dim ,
v) Randomly Create an initial population, ),.....,1;,......,1(, DimjSNijix .
- WHILE ( MCNiter )
i) 1 iteriter
ii) % Employed Bee Phase %
a) FOR ( SNi :1 )
o Produce a new candidate from the existing food source using the
following expression (16).
ikSNkjk
xjixjijixjiv and ,...,2,1 ),,(,,, (8.1)
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 8 Page 50
o Evaluate the quality of the new candidate food source, )( ifit v .
o A Greedy selection algorithm is employed to select one and
discard the other among ix and iv based on their corresponding
fitness values )( ifit x and )( ifit v respectively.
o IF )( ifit v is not better than )( ifit x then the trial counter is
incremented by unity according to the following mathematical
expression (17).
(8.2) 1+= ii trialtrial
END IF
END FOR
iii) % Probability value assignment to each food source %
An onlooker bee chooses a food source depending on the probability value, ip ,
associated with that food source, ix , computed using the following expression
(7.3).
(8.3)
1)(
)(
i
∑=
=SN
nnfit
ifitp
x
x
iv) % Onlooker Bee Phase %
a) 1;1 == iitr
b) WHILE )( SNitr ≤
• 1+= itritr
• Generate a random number )1,0(∈rand
• IF ( iprand < )
o Produce a new candidate from the existing food source using the
following expression (16).
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 8 Page 51
ikSNkjk
xjixjijixjiv ≠∈∀−+= and ,...,2,1 ),,(,,, φ
o Evaluate the quality of the new candidate food source, )( ifit v .
o A Greedy selection algorithm is employed to select one and
discard the other among ix and iv based on their corresponding
fitness values )( ifit x and )( ifit v respectively.
o IF )( ifit v is not better than )( ifit x then the trial counter is
incremented by unity according
1+= ii trialtrial
END IF
END WHILE
c) 1+= ii
d) IF )( SNi >
1=i
END IF
v) % Determine Scout %
a) IF imitltriali >)max(
Replace ix with a new randomly produced solution/food source
END IF
vi) Memorize the global best solution so far
vii) 1+= iteriter
END WHILE
END
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
CHAPTER 9CHAPTER 9CHAPTER 9CHAPTER 9::::
IMPLEMENTED MODEL
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 9 Page 53
Chapter 9
Implemented Model
9. Implemented model:
Fig. 9. Implemented Model
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
CHAPTER 10CHAPTER 10CHAPTER 10CHAPTER 10::::
FUSION RESULTS AND CASE STUDIES
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 10 Page 55
Chapter 10
Fusion Results and Case Studies
10.1. Results and Case Studies:
Now that we have discussed elaborately the algorithm for image fusion we need to implement it
on different component images and prove the stability and robustness of the algorithm. The
results and inferences obtained after extensive experimentation are discussed in detail in this
section.
Case study 1: A thermal image and its corresponding visual image were taken from the
database [7] and, subsequently, the results obtained after implementing the aforementioned
fusion schemes are shown in the following fig. 10.1.1.
Fig. 10.1.1.Results obtained after fusion with different schemes
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 10 Page 56
Now, on closely observing the obtained results as represented by fig. 10.1.1 we can conclude that
by human visual perception the two fused images that are not corrupted with noise and with have
quality edge preservation is obtained by DCTav (1st row, 3
rd column) and by DCTopti (3
rd row,
3rd column). However, on comparing the resultant images of DCTav and DCTopti by SSIM, as a
measure of quality of the fused image, we obtain the following result, as given in TABLE 10.1.
TABLE 10.1
Reference (component) Image SSIM (with DCTav fused
image)
SSIM (with DCTopti fused
image)
Visual 0.54274 0.55721
One can easily infer from the results tabulated in the preceding table 10.1 that the fused image by
DCTopti, the image fusion methodology proposed by us, is far more superior than that obtained
by DCTav [2]. Therefore, the superiority of the image fusion algorithm over one of the
contemporary existing algorithms is proved.
Case study 2: Similarly, another thermal image and its corresponding visual image were taken
from the database [7] and, subsequently, the results obtained after implementing the
aforementioned fusion schemes are shown in the following fig. 10.1.2.
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 10 Page 57
Fig. 10.1.2. Results obtained after fusion with different schemes
Now, on closely noticing the resultant fused images as represented by fig. 10.1.2 one can easily
infer, by human visual perception, that the two fused images that are not corrupted with noise
and have high quality edge preservation is obtained by DCTav (1st row, 3
rd column) and by
DCTopti (3rd row, 3
rd column). However, on comparing the resultant images of DCTav and
DCTopti by SSIM, as a measure of quality of the fused image, we obtain the following result, as
given in TABLE 10.2.
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 10 Page 58
TABLE 10.2
Reference (component) Image SSIM (with DCTav fused
image)
SSIM (with DCTopti fused
image)
Visual 0.61132 0.61836
One can easily infer from the results tabulated in the preceding table 10.2 that the fused image by
DCTopti, the image fusion methodology proposed by us, is far more superior than that obtained
by DCTav [2]. Therefore, again the superiority of the image fusion algorithm over one of the
contemporary existing algorithms is proved.
Case study 3: Now, we need to prove that our algorithm is not domain constrained only to IR
and visual image fusion but also it can result in the fusion of various types of images. One such
example is given here. The input images are taken from the database provided here [32]. The
resultant fused image is shown in the following fig. 8.1.3.
Fig .10.1.3 Results obtained by image fusion using the proposed DCTopti algorithm
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 10 Page 59
On closely observing the two component images, image 1 and image 2, as shown in fig. 10.1.3
we see that in image 1 the camera was focused on the table clock during image acquisition and in
image 2 the camera was focused on the boy during image acquisition. Finally, by the fusion of
the two images image 1 and image 2 using DCTopti, the algorithm proposed by us, we obtain a
fused image where it appears as if the camera was focused both on the table clock and the boy.
Therefore, we can proclaim that the DCTopti image fusion algorithm, proposed by us, performs
well on not only IR and Visual images but also on other types of images.
Case study 4: Now, we use the DCTopti image fusion algorithm, proposed by us, for the
fusion of a visual image, image 1, with a millimeter wave (MMW) image, obtained from the
database in [32], as shown in the following fig. 10.1.4.
Fig.10.1.4. Results obtained by image fusion using the proposed DCTopti algorithm
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 10 Page 60
One can easily infer from the resultant fused image shown in the preceding fig. 1 that our
proposed algorithm can also perform the image fusion of a visual image with a MMW image.
Case study 5: Finally, we test our DCTopti algorithm on two visual images as shown in the
following fig. 8.1.5 and the resultant fused image obtained is quite satisfactory.
Fig.10.1.5. Results obtained by image fusion using the proposed DCTopti algorithm
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 10 Page 61
10.2 .Conclusion:
This research work demonstrates a novel image fusion algorithm for the fusion of Infrared and
Visual image. However, through extensive experimentation it has been proven that this algorithm
can also perform the operation of image fusion over other different types of images. One of the
salient aspects of this research work is that it uses the renowned Artificial Bee Colony algorithm
for optimization of a novel cost function for determining the fusion weights for the DCT
coefficient matrix of two different image patches. Not only by human visual perception but also
by the SSIM score it has been proven that the algorithm proposed by us outperforms most of its
other contemporary competitors. The authors intend to undertake the segmentation of desired
objects from the fused images as a future prospect of research work.
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
CHAPTER 11CHAPTER 11CHAPTER 11CHAPTER 11::::
OVERVIEW OF IMAGE SEGMENTATION
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 11 Page 63
Chapter 11
Overview of Image Segmentation
11.1. Challenges for Image Segmentation:
Image segmentation, i.e. partitioning an image into homogeneous areas, is one of the most
fundamental problems in a variety of applications, including but not limited to remote sensing,
optical imaging, and medical image analysis. The level set based Chan-Vese algorithm primarily
uses region information for successive evolutions of active contours of concern towards the
object of interest and, in the process, aims to minimize the fitness energy functional associated
with. Orthodox gradient descent methods have been popular in solving such optimization
problems but they suffer from the lacuna of getting stuck in local minima and often demand a
prohibited time to converge. Although great strides have been made toward a general
segmentation scheme, many difficult challenges still exit for this problem, especially on medical
images. For example, poor image contrast and noise are very common for many modalities, such
as ultrasound, positron emission tomography (PET) and single photon emission computed
tomography (SPECT). Patient movement and partial volume effect in imaging process can easily
further deteriorate the image quality by blurring tissue boundaries. It is because of this weakness
in the current technology that leads us to propose a new segmentation method that does not stand
alone, but relies on prior information about the shape of interest. This reliance on previous
knowledge of generate acceptable results in segmentation is especially applicable in the domain
of medical images because it is most often the goal of the physician to obtain the segmentation of
a particular region or object. Combining the knowledge of what items are to be found with the
knowledge of what shape those items have possessed in the past allows our method to accurately
and efficiently and segment images that would otherwise be impossible to do without
intervention.
In computer vision, segmentation is the process of partitioning a digital image into multiple
segments. The goal of segmentation is to simplify and change the representation of an image into
something that is more meaningful and easier to analyze. Image segmentation is typically used
to locate objects and boundaries in images.
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 11 Page 64
11.2. Overview of Segmentation Algorithms:
Image Segmentation is the process of finding a mapping σ such that:
,2: iΡ→Ζσ ( )Ν≤≤ i1 (11.2.1)
Here 1Ρ , ,......,
2Ρ
NΡ are the Ν Classes to which a pixel can belong. Then we can define a
few useful notions for Segmentation. W e denote the original image as .Ι
Ι=Ρ=i
N
iU1
(11.2.2)
( ) Φ=Ρ∩Ρ→≠∀ jijiji, (11.2.3)
Equation 11.2.2 signifies that every pixel is contained in at least one class such that the union of
all the sets yields the entirety of the original image. Next stipulation, which found in equation
11.2.3, says that no pixel can belong to more than one class and consequently, the intersection of
any two different classes is necessarily the empty setϕ .
There are many different methods for arriving at segmentation and of determining the worth of
Segmentation. The Segmentation algorithms can be classified into three categories:
(a) Pixel-based Algorithms
(b) Edge-based Algorithms
(c) Region-based Algorithms
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
CHAPTER 12:
GSA-K MEANS CLUSTERING CHEN AND VESE
SEGMENTATION
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 12 Page 66
Chapter 12
GSA-K Means Clustering Chen and Vese
segmentation
12.1 Chan and Vese
12.1.1. Introduction:
Active Contour Models [58, 66, 77] also called snakes are used for detecting an object outline
from an image. Very often the active contour models employ energy based segmentation
techniques [68], where the basic idea is to minimize the energy associated with the active
contour, as the curve evolves to fit around the desired objects. The energy associated with an
active contour generally consists of internal energy and external energy. The internal energy
deals with the properties of the contour such as area enclosed, length of the contour and its
smoothness. The external energy depends upon the image structure and the user imposed
constraints.
Chan-Vese (C-V) model is a very standard active contour based approach in image
Segmentation. This model uses region-based information in its level set based formulation and
tries to minimize an energy fitting functional associated with it by solving an Partial Differential
Equation. Though this model can segment internal objects very well, it still suffers from the
problem of getting stuck at local minimum due to the fitness functional being a non-convex and
non-unique one. Often different initial contours give varied segmentation results.
A unique method developed by Gibou and Fedkiw [59] was developed that solves the C-V
model by using the k_means clustering algorithm. Although this method [58] was found to be
much faster than the standard C-V model and did not require the need for level sets, it still
suffers from the problem of getting stuck at local minimum if the initial cluster points are not
appropriately selected. This error increases as the number of classes into which the image has to
be segmented increases.
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 12 Page 67
In this work, evolutionary algorithms have been used in conjunction with the k_means algorithm
to take care of this problem. Successful implementations have been done using the Gravitational
Search Algorithm (GSA) [60] to segment images into two-class, three-class and four-class
effectively. Our model is seen to outperform the model suggested by Gibou and Fedkiw [62].
12.1.2. The Chan and Vese Model (Piecewise constant model) for Image
Segmentation:
As mentioned before, Chan and Vese [61] proposed a piecewise-constant model for image
segmentation. An evolving curve C in Ω , is defined as the boundary of an open subset ω of Ω (i.e. Ω⊂ω , and ω∂=C ). Then )(Cinside denotes the regionω , and )(Coutside denotes the
region ω/Ω [61]. The image 0u is assumed to be formed by two regions of piecewise-constant
intensities having distinct values i
ou and ou0 . i
ou represents the intensity of the object to be
detected and ou0 the intensity of the background of the object. The object is assumed to have a
boundary or bounding contour 0C . Then the intensity inside 0C is i
ou and the intensity outside
0C is ou0 . Thus the energy fitting term can be defined as [61]:
dxdycyxu
dxdycyxu
CinsideAreaCLengthCccF
Coutside
Cinside
2
)(202
2
)(101
21
),(
),(
))((.)(.),,(
∫∫
−+
−+
+=
λ
λ
µν
(12.1.2.1)
where C is any curve that is being iteratively evolved, and the constants 21 and cc denote the
average intensity of the image inside and outside the evolving curve C . The free parameters of
the equation in (12.1.2.1) must all be positive. The fitting energy (12.1.2.1) also contains some
penalizing terms such as the length of the evolving curve C and the area of the region inside the
curve C . These two terms helps to smoothen the evolving contourC . It is quite evident from the
above equation (12.1.2.1) that the energy associated with the contour C becomes minimum
when 0CC ≈ i.e., the evolving contour sits exactly on the object boundary.
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 12 Page 68
The above functional in (12.1.2.1) is quite similar to that as the Mumford-Shah Functional [63].
The Mumford-Shah Functional (MSF ) tries to segment an image into its various sub-regions by
using region-based information. The Chan and Vese model tries to represent each region or
connected component iR of C/Ω using a constant intensity ic . This ic represents the average
intensity of the region i.e. )( 0uaverageci = on each connected component iR [61, 63]. This
reduced case is called the minimal partition problem. Here the values of the constants are:
))((
))((
2
1
Coutsidemeanc
Cinsidemeanc
=
=
(12.1.2.2)
12.1.3. Level Set Formulation of the Model:
In level set methods [61, 69], a contour Ω⊂C is represented by the zero level set of a Lipschitz
function R→Ω:φ . This is also called a level set function and it is defined in such a way that
<Ω∈=Ω=
>Ω∈==
=Ω∈=∂=
0),(:),(/)(
0),(:),()(
0),(:),(
yxyxCoutside
yxyxCinside
yxyxC
ωφω
φω
(12.1.3.1)
Using the level set function φ and also the Heaviside function H and the one-dimensional
Dirac measure δ0 so as to use one-dimensional calculations, the energy fitting terms ),,( 21 CccF
can be reformulated as [61]:
dxdyyxHcyxu
dxdyyxHcyxu
dxdyyxHdxdyyxyxccF
))),((1(),(
)),((),(
)),((),()),((),,(
2
202
2
101
21
φλ
φλ
φµφφδνφ
−−−−−−−−++++
−−−−++++
++++∇∇∇∇====
∫∫∫∫∫∫∫∫
∫∫∫∫∫∫∫∫
Ω
Ω
ΩΩ
(12.1.3.2)
Now, one can determine the constants c1 and c2 by keeping φ fixed and minimizing the energy
functional ),,( 21 φccF with respect to c1 and c2. Hence c1 and c2 can be expressed in terms of φ ,
given as [61]:
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 12 Page 69
∫
∫
Ω
Ω=dxdyyxH
dxdyyxHyxu
c)),((
)),((),(
)(
0
1φ
φφ
(12.1.3.3)
∫
∫
Ω
Ω
−
−
=dxdyyxH
dxdyyxHyxu
c))),((1(
))),((1)(,(
)(
0
2
φ
φφ
(12.1.3.4)
Here, (12.1.3.3) can be utilized, provided 0)),(( >∫Ω
dxdyyxH φ , and (12.1.3.4) can be utilized,
provided 0))),((1( >−∫Ω
dxdyyxH φ . In [61], regularized (smooth) versions of the Heaviside
function H and the one-dimensional Dirac function 0δ , called εH and εδ respectively, are
utilized to compute the Euler-Lagrange equations associated with the computation of φ and they
are defined as:
22
' 1)()(arctan
21
2
1)(
xxHx
xxH
+==
+=ε
επ
δεπ εεε
(12.1.3.5)
Keeping the constants 21 ,cc fixed, and minimizing F with respect to φ , the associated Euler-
Lagrange equation for φ is deduced. This finally gives the following equations for the curve
evolution which must be solved iteratively:
( ) ( ) 0)(2
202
2
101 =
−+−−−
∇∇
=∂∂
cucudivt
λλµφφ
νφδφ
ε
(12.1.3.6)
Ω= in),(),,0( 0 yxyx φφ
(12.1.3.7)
Ω∂=∂
∂
∇on0.
)(
nrφ
φφδε
(12.1.3.8)
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 12 Page 70
It is also very important to remember that as the evolving curve can develop shocks and degrade
with time, it is necessary to periodically reinitialize the zero level curve of φ to the signed
distance function [59, 67, and 71]. The convention is to solve the following equation periodically
so as to re-initialize the level set, where φ the function to be re-initialized is and )(φsign is the sign
function.
( )φφφ
∇−=∂∂
1)(signt (12.1.3.9)
However, it should be remarked in this context that, often the step of re-initialization introduces
further complications like moving the zero level set away from its original location or increasing
the computation time. In fact, it is not really well established yet, when and how such a re-
initialization step should be implemented [71]. Algorithm 1 presents the basic Chan and Vese
algorithm.
12.1.4 The Chan and Vese Algorithm:
The st.eps of the basic Chan and Vese algorithm [61] can be summarized as given in algorithm.
12.1.4.1
______________________________________________________________________________
______________
Step 1. Construct the initial level set function 0φ for iteration n = 0.
Step 2. Calculate the values of the average intensities inside and outside the evolving contour
by computing
)(1nc φ and nc )(2 φ , using (12.1.3.3) and (12.1.3.4) respectively, at iteration = n.
Step 3. Solve the Partial Differential Equation in φ (12.1.3.6), (12.1.3.7), (12.1.3.8) to obtain
the new level set function 1+nφ for
iteration = n+1.
Step 4. The level set function φ may have to be reinitialized locally using (12.1.3.9). This step is optional and, if
Employed, is generally repeated after every few iterations of the curve evolution.
Step 5. Compare the level set functions ( nφ , 1+nφ ). If the solution is not stationary, Then make
n = n+1 and
go to Step 2, Otherwise go to Step 6.
Step 6. Stop contour evolution and report segmentation result.
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 12 Page 71
______________________________________________________________________________
_______________
Algo. 12.1.4.1. The Chan and Vese algorithm for image segmentation using level set approach.
12.1.5. The Strengths and Drawbacks of the Chan and Vese Algorithm:
The inherent strength of the above mentioned method is that interior contours within the image
can be automatically detected using this method [61]. However, sometimes, if an initial contour
is placed far away from the final segmented result, it may be so that this method will not be able
to converge to the final result within the desired number of iterations. Also, the energy fitting
functional term is non-convex in nature [64, 65] and contains several local minimum points.
Since this method utilizes a gradient descent based adaptation procedure, good initial choices of
the contour are essential as, otherwise, the system may get locked at the first local minimum of
the functional. So if the initial contour can be placed in a near optimal position, it will be able to
achieve the final segmented result quickly, within a small number of iterations. The optimal
placement of the initial contour will also be able to avoid the local minimum and converge closer
to the global minimum point. As mentioned previously, to overcome this initialization problem
of the Chan and Vese algorithm, we propose a metaheuristic optimization based robust
initialization procedure that can finally achieve optimum or near optimum contour(s),
irrespective of the choices of the initial contour.
Another feature of the Chan and Vese algorithm that requires discussion is the (optional) re-
initialization step. This re-initialization step is computationally expensive and a better way to
evolve the curve is to force the evolving contour to be as close to a signed distance function as
possible. Using the property of the signed distance function [71], a metric as in (12.1.5.1) can be
added to the energy fitting term in (12.1.2.1). By adding this term, now (12.1.5.3) can be solved
iteratively without the need for any re-initialization of level set function. Under these
circumstances, there will be no need to implement step 4 in algo. 12.1.4.1. The mathematical
formulation can then be described as [71]:
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 12 Page 72
( ) dxdyP ∫ −∇=2
12
1)( φφ
(12.1.5.1)
)(),2
,1
()2
,1
,( φφφ PccFccE +=
(12.1.5.2)
( ) ( )
−+−−−
∇∇
+
∇∇
−∇=∂∂ 2
202
2
101
2 )( cucudivdivt
λλµφφ
νφδφφ
φφ
ε
(12.1.5.3)
12.1.6 Extension of Chan Vese algorithm for Vector-valued images:
The Chan and Vese model can be extended to cater to the vector-valued images (such as RGB or
multi-spectral images) [70]. The model is similar to the scalar model of the Chan Vese
algorithm. The only difference is that the model minimizes the fitting energy over each
component of the vector-valued images. This model also has very strong de-noising capabilities.
Let i
u,0 be the i th channel of an image onΩ , with Ni ,....,3,2,1= channels and C be the
evolving curve. The constant vectors are defined as ),......,2
,1
( +++=+N
cccc and
),......,2
,1
(−−−=−N
cccc [70]. Then similar to (12.1.2.1), for the vector-valued case, the energy
functional can be given as [70]:
∫ ∑
∫ ∑
=
−−
=
++
−+
−
−+
+=
)( 1
2
,0
)( 1
2
,0
),(1
),(1
))((.)(.),,(
Coutside
N
i
iii
Cinside
N
i
iii
dxdycyxuN
dxdycyxuN
CinsideAreaCLengthCF
λ
λ
µνcc
(12.1.6.1)
Where +iλ and −
iλ are the parameters for the ith channel and must be positive.
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 12 Page 73
The constant +c and −
c vectors represent the average image intensities, inside and outside the
evolving contourC , for each channel of the vector-valued image. Similar to the scalar case, it is
prudent to use level set formulation for minimizing the above functional for the vector valued
case too. Also as given before, regularized (smooth) version of the Heaviside function εH and
the one-dimensional Dirac function εδ are used in the formulation.
Minimizing the energy in (12.1.6.2) with respect to the constants +c and −
c , for Ni ,....,3,2,1= ,
we obtain [70]:
∫
∫
Ω
Ω+ =dxdyyxH
dxdyyxHyxu
c
i
i
)),((
)),((),(
)(
,0
φ
φφ
∫Ω
−
∫Ω
−=−
dxdyyxH
dxdyyxHyxi
u
ic ))),((1(
))),((1)(,(,0
)(φ
φφ
(12.1.6.2)
The extra regularizing term in (12.1.5.1) is added as in the scalar case to (112.1.6.2) to avoid the
necessity of the additional re-initialization phase. Similarly the energy functional in (12.1.6.2.)
can be minimized by the gradient descent search, with respect to φ by keeping the vectors +c and
−c constant. The following Euler-Lagrange equation has been derived that needs to be solved iteratively
so as to evolve the curve [70]:
∑=
−−−+∑=
+−+−−∇
∇+
∇
∇−∇=
∂
∂ N
iicyxiui
N
N
iicyxiui
Ndivdiv
t 1
2),(,0
1
1
2),(,0
1)(
2 λλµφ
φνφεδ
φ
φφ
φ
(12.1.6.3)
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 12 Page 74
12.2 K-means Clustering Algorithm
The main concept of clustering is to make a group of similar data in to cluster that
comprises of almost similar members [73]. All the members in a group are different in
comparison with other members in that group. K indicates the number of clusters and the
value of K is positive. In this work, the value of K is set to 2. The training of this algorithm
completes, when there is no such change in any cluster.
TABLE 12.1. K-means clustering Algorithm [74]
1. Inputs: number of K and dataset for intrusion detection.
2. Outputs: Set of clusters K whish minimize square-error criterion
3. Initialization: Select K elements for data randomly and initialize K clusters.
4. Repeat step 2, when number of cluster structure changes.
5. Cluster determination: To which source data belongs. Add element to the cluster
with minimum (Using Euclidean distance) Distance ( iQiP , )
6. Mean calculation: Mean of cluster. Using step 3, change in cluster centroid to mean
obtained.
12.3. GSA:
GSA optimization algorithm is based on law of gravity [75]. Newton defined it as, “Every
particle in the universe attracts every other particle with a force that is directly proportional
to the product of the masses of the particles and inversely proportional to the square of the
distance between them.”
System having G masses in which the j thmass’s position is defined as:
S Gjgjs
djsjsjsj ,....,2,1),,.....,,......,2,1( ==
(12.3.1)
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 12 Page 75
where, d
js is jth mass’s position in d th dimension and in the search space, g is total
numbers of dimensions. The calculated mass of each agent:
∑ = −
−=
Gk
tworsttkfit
tworsttjfittj
O
1)()(
)()( (12.3.2)
Where, tj
O is the mass and )(tj
fit is fitness value of agent j at t.
and ,...,1),(max)( Gktk
fittworst ∈=
(12.2.1)
TABLE.12.2. GSA K-means approach Algorithm [76]
1. Scaled dataset
2. For K =2.
Using Euclidean distance
Centroid determination of each cluster
3. For producing 2 centroids
4. Determine number of agents
Define G’ constant
Compute fitness function
Stopping condition is [0,100]
5. Resultant GSA K-means
12.4. Proposed HGSA_K-means based CV model:
Proposed GSA-K-means based CV model is presented here. In this approach, the fitting
functional energy of the CV model is used to assign masses to the agents in the search
space. The algorithm is described below:
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 12 Page 76
The basic method is as follows: Suppose the motive is to segment the image into three-
parts. After the first step, the image has been divided into two clusters – one object and its
complementary background. The intensity variation across the captured object and its
complementary background is compared, and the one having the larger variation becomes
the target for the next step of the hierarchy. The intensity with the smaller intensity
variation is not used for the next step of segmentation. Thus the image will be divided into
three distinct parts using the modified proposed CV model. Figure 12.4.1 describes the
Binary tree structure of hierarchical segmentation algorithm.
Figure 12.4.1. Binary tree structure of hierarchical segmentation
The procedure is represented as a binary tree with the regions being represented as nodes
and children of each node being created by a single step of the two-cluster GSA_K
algorithm. The set of the leaves of this tree constitutes the final segmentation of the
original image into a number of clusters equal to the number of leaves.
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 12 Page 77
12.5. Simulation Results and Discussions:
This proposed method is implemented in Mat lab (version R2013a, using Core 2 Duo CPU, 2.66
GHz, 3GB RAM). In this work, hierarchical setup is used to continuously segment the images
following the Chan-Vese Image segmentation model. For this approach, HGSA_k-means
clustering based CV model is used for image segmentation. To illustrate the effectiveness of our
method we use the PCA fused images for segmentation as their segmentation results are required
to be accurate. We use four sample images to segment them into two, three and four class
respectively. For each image, we have given five test runs of both the standard k_means based C-
V model (G-F) and our proposed GSA-k_means based C-V model (GSA-k) and noted down the
results.
(a) (b) (C) (d)
Fig.12.5.1. Sample PCA fused images (a, b, c, d) on which the experiment has been done
Finally, Figure 12.5.2.Gives the segmentation result of this HGSA_K-means proposed
algorithm when applied on the sample images of figure 4 in its 2-class implementation. All
of the images are of size256x256.
Here, it is also reported about the progression of error while detecting the clusters in the
image for segmentation.As it is evident from the above observation that the percentage of
error increases as the number of clusters to be increased.
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 12 Page 78
Fig12.5.2: Two-class segmentation of the sample images [7] of figure: 10.5.1 by this
proposed algorithm. Here, k-mean-iteration=5, max-it=5; and N=5.
TABLE 12.3.Timing comparison between the standard C-V model and our proposed C-V
model for segmentation of some gray-scale images of size 256x256(as present in fig
.12.5.2).
Images C-V Model(in
seconds)
Our Proposed C-V
Model(in seconds)
(a) 3.21 0.79
(b) 3.11 0.28
(c) 2.42 0.56
(d) 2.39 0.21
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 12 Page 79
Here, we also report the progression of error while detecting the clusters in the image for
segmentation. As it is quite evident from the above observations,the error percentage increases as
the number of clusters to be detected increases.
Cluster Value
Mean average standard deviation GF model
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Image (a) Image (b) Image (c) Image (d)
2 class
3 class
4 class
Images 4 class 3 class 2 class
Image (a) 30.45768 13.67835 18.67457
Image (b) 13.45631 1.899138 32.85679
Image (c) 20.36278 3.435678 34.78345
Image (d) 18.42367 1.823498 37.38236
Fig.12.5.3. Mean average standard deviation of the GF model. Here the deviation from the
nominal value is quite high which shows that this method often diverges from the global
minimum value.
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 12 Page 80
Cluster Value
Mean average standard deviation GSA_K model
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Image (a) Image (b) Image (c) Image (d)
2 class
3 class
4 class
Images 4 class 3 class 2 class
Image (a) 0.149071 0 0.341267
Image (b) 1.452897 0.798327 0
Image (c) 0 0.182574 2.365891
Image (d) 0 15.36159 0
Fig.12.5.4. Mean average standard deviation value from our proposed model. The deviation is
relatively kept under check for all the class divisions for image (a)-(b)-(d). However, for the
image (c), the deviation is quite high for the 2-class image segmentation by our proposed model.
As we can see from the above observations that the clustering algorithm’s performance
decreases steadily as the number of clusters is increased. In proposed HGSA_K method,
the error percentage is very low for the 2 class cases. So it seems prudent enough to
convert the multi-cluster problem into a subsequent iterative based 2-cluster problem so as
to decrease this error.
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 12 Page 81
The advantages of proposed method are as follows:
1. No need for level sets.
2. Good convergence towards the global minimum.
3. Faster method than the normal CV model.
4. At each step, only the relevant parts of the image is being used for clustering based
segmentation, thereby, preventing lots of redundant calculations and saving
computation time
Performance:
The segmentation performance is also evaluated quantitatively where the automatically
segmented image is compared with the manually segmented image and the similarity is
expressed in terms of Segmentation Performance Measure (SPM), expressed in percentage.
Table 12.4 shows this performance for our proposed system, for each of the four independent
runs of our algorithm with different initial choices, as mentioned before. Ideally the SPM should
be as high as possible with its maximum possible value being 100%. It can be seen that, in each
case, our proposed algorithm could achieve an SPM of more than 99.925%, which should be
considered as a highly encouraging performance.
TABLE 12.4:The segmentation performance of our proposed algorithm for fig. 12.5.2.
a 99.8298
b 99.9175
c 99.9662
d 99.9865
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 12 Page 82
12.6. Conclusions:
Our proposed GSA-k_means algorithm is successful in segmenting images by using the C-
V model. Due to the coupling of the standard k_means clustering algorithm along-with the GSA
algorithm, the convergence of the cluster values to the global minimum is almost always
guaranteed. It is also necessary to point out that this implementation does not need the use of
level sets and is much quicker than the standard level set implementation of the C-V model.
Extensive experiments in two-class image segmentations have been performed and it has been
observed that our method outperforms both the standard C-V model and the k_means G-F
model. Our proposed C-V model is much faster than the standard C-V model and is also
insensitive to the contour initializations. Experiments done on some fused images validate our
claim. For future work, our model can be extended towards multi-phase level set segmentation
and also for vector valued images. Proposed HGSA_K-means CV algorithm is successful in
segmenting images by using the CV model. Due to the coupling of the standard K-means
clustering algorithm along-with the GSA algorithm, the convergence of the cluster values to the
global minimum is almost always guaranteed.
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
CHAPTER 13CHAPTER 13CHAPTER 13CHAPTER 13: : : :
CONCLUSION & FUTURE SCOPE
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 13 Page 84
Chapter 13
Conclusion & Future Scope
13. Conclusion:
This chapter describes the conclusion about the research work and gives the suggestion for
future work.
13.1 Conclusion:
In this research work, attention was drawn towards the current trend of the use of Infrared and
Visual image fusion techniques, especially approaches based on Discrete Cosine Transforms.
This research work demonstrates a novel image fusion algorithm for the fusion of Infrared and
Visual image. However, through extensive experimentation it has been proven that this algorithm
can also perform the operation of image fusion over other different types of images. One of the
salient aspects of this research work is that it uses the renowned Artificial Bee Colony algorithm
for optimization of a novel cost function for determining the fusion weights for the DCT
coefficient matrix of two different image patches. Not only by human visual perception but also
by the SSIM score it has been proven that the algorithm proposed by us outperforms most of its
other contemporary competitors. The authors intend to undertake the segmentation of desired
objects from the fused images as a future prospect of research work.
Our proposed C-V model is much faster than the standard C-V model and is also insensitive to
the contour initializations. Experiments done on some fused images validate our claim. For
future work, our model can be extended towards multi-phase level set segmentation and also for
vector valued images. Proposed HGSA_K-means CV algorithm is successful in segmenting
images by using the CV model. Due to the coupling of the standard K-means clustering
algorithm along-with the GSA algorithm, the convergence of the cluster values to the global
minimum is almost always guaranteed. Our proposed GSA-k_means algorithm is successful in
segmenting images by using the C-V model. Due to the coupling of the standard k_means
clustering algorithm along-with the GSA algorithm, the convergence of the cluster values to the
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
Chapter 13 Page 85
global minimum is almost always guaranteed. It is also necessary to point out that this
implementation does not need the use of level sets and is much quicker than the standard level
set implementation of the C-V model.
13.2 Future Scope:
As a future work we intend to consider the use of the second order partial derivatives of the
cost function for faster and better convergence of the active contour model. For future work, our
model can be extended towards multi-phase level set segmentation and also for vector valued
images. Proposed HGSA_K-means CV algorithm is successful in segmenting images by using
the CV model in its both two-phase and multi-phase implementation. Due to the coupling of the
standard K-means clustering algorithm along-with the GSA algorithm, the convergence of the
cluster values to the global minimum is almost always guaranteed. From segmentation to image
imprinting and denoising problems and beyond, such methods will no doubt play an important
role in future image analysis research work and try to implement it by hardware. The final aspect
in future development and improvement is how to estimate and evaluate the quality of a fused
image. As we have discussed in previous and it develop by hardware applications. A few, right
to use methods without reference image are significant for our concern in Infrared and Visual
imaging system.
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
REFERENCES
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
References Page 87
REFERENCES
1. Zheng Lio, Erik Blasch, ZhiyunXue, Jiying Zhao, Wei Wu,“Objective assessment of
multiresolutionimagfusion algorithm for context enhancement in night vision: a comparative study,”
IEEE Transaction on Pattern Analysis and Machine Intelligence,vol.34,no.1, pp. 94-109, January 2012.
2. VPS Naidu,“Discrete Cosine Transform based Image Fusion Techniques,” Journal of Communication
Navigation and Signal Processiog,vol.1,no.1,pp.35-45,January 2012.
3. ThiThiZin, Hideya Takahashi, Takashi Toriu and Hiromitsu Hama, “Fusion of infrared and visible
images for robust person detection,” Image Fusion , Chapter 12, pp.239-264.
4. A.Toet, M.A.Hogervorst, S.G.Nikolov, J.J.Lewis, T.D.Dixon, D.R.Bull, C.N.Canagarajah,“Towards
cognitive image fusion” Information Fusion 11 (Springer-Verlag, 2010), pp.95-113.
5. Zhou Wang, Alan Conrad Bovik, Hamid Rahim Sheikh, and Eero P. Simoncelli, “Image Quality
Assessment: From Error Visibility to Structural Similarity,” IEEE Transactions on Image Processing, ol.
13, no. 4, April 2004, pp. 600-612.
6. Alan C. Brooks and Thrasyvoulos N.Pappas,“Structural similarity quality metrics in coding context:
exploring the space of realistic distortion,” IEEE Transaction on Image
Processing,vol.17,issue.8,pp.1261-1273,Aug.2008.
7. OTCBVS Benchmark Data Collection, http://www.cse.ohio-state.edu/otcbvs-bench/
8. Ujwala Patil and Uma Mudengudi, “Image Fusion Using Hierarchical PCA,” In: The International
Conference on Image Information Processing, pp. 1-6. IEEE Press, Himalchal Pradesh (2011)
9. N. Ahmed, T. Natarajan and K. R. Rao, “Discrete Cosine Transform”, IEEE Trans. On Computers, Vol.
32, pp. 90-93 (1974)
10. Vassil Dimitrov and Khan Wahid, “Multiplier less DCT Algorithm for Image compression
Applications,” International journal on Information Theories and Application, Vol.11, pp.162-169,2004.
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
References Page 88
11. Musrrat Ali, Chang Wook Ahn, Millie Pant, “A robust image watermarking technique using SVD and
differential evolution in DCT domain”, Optik125 (Springer-Verlag, 2014), pp.428-434 (2014).
12. Phen-Lan Lin, Po-Ying Huang, “Fusion methods based on dynamic-segmented morphological Wavelet
or cut and paste for multifocus images”, Signal Processing88 (Springer-Verlag, 2008) ,pp. 1511-1527
(2008)
13. I.Bloch, Information Combination Operators for data fusion : a review with classification, IEEE
Trans.SMC: Part a 26 (1996) 52-67.
14. Yong Jiang, Minghui Wang,“ Image fusion with morphological Component analysis,” Information
Fusion 18(Springer-Verlag,2014),pp.107-118.
15. Fouzi Douak, Redha Benzid, Nabil Benoudjit,“Color Image Compression algorithm based on the DCT
transform Combined to an adaptive block scanning”, Int. j. Electron. Commun. (AEU)65 (Springer-
Verlag,2011), pp.16-26(2011).
16. S. Rashidi, A. Fallah, F. Towhidkhah,“ Feature extraction based DCT on dynamic Signature
Verification”, Scienttia Iranica D 19(6) (Springer,2012),pp.1810-1819(2012).
17. R. C. Gonzalez and P. Wintz,“ Digital Image Processing”, Ma: Addison-Wesley, 1987.
18. G. Strang ,“ The Discrete Cosine Transform”, SIAM Review, Vol.41, pp.135-147,1999.
19. Jan Flusser, Filip Sroubek, and Barbara Zitova,“ Image Fusion Principles, Methods, and Applications”,
Tutorial Eusipco (2007)
20. Wang, j.; Liang, j.; H.; Li, Y; Feng, B. (2007) Performance evaluation of infrared and visible image
fusion algorithms for face recognition, proceedings of international conf. intelligent system and
knowledge Engineering (ISKE2007), pp. 1-8, 2007.
21. Leykin Alex, Ran Yang, Hammoud Riad.(2007). Thermal-Visible Video Fusion for Moving Target
Tracking and pedestrian Classification, IEEE conference on computer vision and Pattern Recognition,
pp.1-8.
22. D. Karaboga, “An idea based on honey bee swarm for numerical optimization,” Techn. Rep. TR06,
Erciyes Univ. Press, Erciyes, 2005.
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
References Page 89
23. D. Karaboga, and B. Basturk, “On the performance of artificial bee colony (ABC) algorithm,” Applied
Soft Computing, Elsevier, vol. 8, Issue 1, pp. 687-697, January 2008.
24. D. Karaboga, and B. Akay, “A comparative study of artificial bee colony algorithm,” Applied
Mathematics and Computation, Elsevier, vol. 214, Issue 1, pp. 108-132, August 2009.
25. E. Hancer, C. Ozturk, and D. Karaboga, “Artificial Bee Colony based Image Clustering Method,” in
Proc. 2012 IEEE Congress on Evolutionary Computation (CEC), pp. 1-5, June 2012.
26. T. R. Benala, S. D. Jampala, S. H. Villa, and B. Konathala, “A Novel approach to Image Edge
Enhancement using Artificial Bee Colony optimization algorithm for hybridized smoothening filters,” in
Proc. 2009 World Congress on Nature & Biologically Inspired Computing(NaBIC), pp. 1071-1076,
December 2009.
27. H. Zhihui, Y. Weiyu, L. Shanxiang, and F. Jiuchao, “Multi-Level Threshold Image Segmentation using
Artificial Bee Colony algorithm,” in Proc. 2013 Fifth International Conference on Measuring
Technology and Mechatronics Automation (ICMTMA), pp. 707-711, January 2013.
28. D. Karaboga,and C. Ozturk, “neural Networks training by Artificial Bee Colony algorithm on Pattern
Classification,” Neural Networks World 19, no. 3, pp. 279-292, 2009.
29. F. Jin, and G. Shu, “Path Planning of Free-flying Space Robot Based on Artificial Bee Colony
Algorithm,” in Proc. 2012 2nd International Conference on Computer Science and Network Technology
(ICCSNT), pp. 505-508, December 2012.
30. S. Biswas, A. Chatterjee, and S. K. Goswami, “An artificial bee colony-least square algorithm for solving
harmonic estimation problems,” Applied Soft Computing, Elsevier, vol. 13, Issue 5, pp. 2343–2355, May
2013.
31. Z.Wang and A.C.Bovik, “A Universal Image Quality Index,” IEEE Signal Processing
Letters,vol.9,no.3,pp.81-84,Mar.2002.
32. http://www.ece.lehigh.edu/SPCRL/IF/image_fusion.htm
33. Desale, Rajenda Pandit, and Sarita V. Verma. “Study and analysis of PCA, DCT & DWT based image
fusion techniques." In Signal Processing Image Processing & Pattern Recognition (ICSIPR), 2013
International Conference on, pp. 66-69. IEEE, 2013.
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
References Page 90
34. Prakash, Chandra, S. Rajkumar, and P. V. S. S. R. Mouli. "Medical image fusion based on redundancy
DWT and Mamdani type min-sum mean-of-max techniques with quantitative analysis." In Recent
Advances in Computing and Software Systems (RACSS), 2012 International Conference on, pp. 54-59.
IEEE, 2012.
35. Mohamed, M. A., and B. M. El-Den. "Implementation of image fusion techniques for multi-focus
images using FPGA." In Radio Science Conference (NRSC), 2011 28th National, pp. 1-11. IEEE,
2011.
36. Haghighat, Mohammad Bagher Akbari, Ali Aghagolzadeh, and Hadi Seyedarabi. "Real-time fusion of
multi-focus images for visual sensor networks." In Machine Vision and Image Processing (MVIP), , pp.
1-6. IEEE, 2010.
37. Pei, Yijian, Huayu Zhou, Jiang Yu, and Guanghui Cai. "The improved wavelet transforms based image
fusion algorithm and the quality assessment." In Image and Signal Processing (CISP), 2010 3rd
International Congress on, vol. 1, pp. 219-223. IEEE, 2010.
38. Li, Hui, B. S. Manjunath, and Sanjit K. Mitra. "Multisensor image fusion using the wavelet
transforms." Graphical models and image processing , vol. 3,pp. 235-245. IEEE 1997.
39. He, D-C., Li Wang, and Massalabi Amani. "A new technique for multi-resolution image fusion." In
Geoscience and Remote Sensing Symposium, vol. 7,pp.4901-4904.IEEE,2004.
40. Wang, Qiang, and Yi Shen. "The effects of fusion structures on image fusion performances. In
Instrumentation and Measurement Technology Conference, 2004. IMTC 04. Proceedings of the 21st
IEEE, vol. 1, pp. 468-471. IEEE, 2004.
41. T.Zaveri, M.Zaveri, V.Shah and N.Patel. “A Novel Region Based Multifocus Image Fusion Method.” In
Digital Image Processing, International Conference on, pp.50-54.IEEE,2009.
42. O.Rockinger. “Image sequence fusions using a shift-invariant wavelet transform.” In image processing ,
1997 International Conference on, vol. 3, pp. 288-291. IEEE1997.
43. Ghimire Deepak and Joonwhoan Lee. “Nonlinear Transfer Function-Based Local Approach for Color
Image Enhancement.” In Consumer Electronics, 2011 International Conference on, pp. 858-865.
IEEE,2011.
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
References Page 91
44. Sruthy, S., Latha Parameswaran, and Ajeesh P. Sasi. "Image Fusion Technique using DT-CWT." In
Automation, Computing, Communication, Control and Compressed Sensing, International Conference
on, pp. 160-164. IEEE, 2013.
45. Patil, Ujwala, and Uma Mudengudi. "Image fusion using hierarchical PCA,." In image Information
Processing (ICIIP), International Conference on, pp. 1-6. IEEE, 2011.
46. Aribi, Walid, Ali Khalfallah, Med Salim Bouhlel, and Noomene Elkadri. "Evalua tion of image fusion
techniques in nuclear medicine," In Sciences of Electronics, Technologies of Information and
Telecommunications (SETIT), 2012 6th International Conference on, pp. 875-880. IEEE, 2012.
47. Pohl et al. “Multi –sensor image fusion in remote sensing : concepts and application “int. j. of Remote
sensing, vol.19, no.5, pp.823-854, 1998.
48. A. Katartzis and M. Petrou,“Robust Bayesian estimation and normalized convolution for super-resolution
image reconstruction”, Workshop on Image Registration and Fusion Computer Vision and Pattern
Recognition, CVPR’07, Minneapolis, USA, pp.1-7, June 2007.
49. N.Mitianoudis and T. Stathaki “Image fusion schemes using ICA bases”, Information fusion 8, pp.131-
142, 2007.
50. Gibou, F., Fedkiw, R.: A Fast Hybrid k-Means Level Set Algorithm For Segmentation. In: Proc. Fourth
Annual Hawaii International Conference on Statistics and Mathematics.
51. Morse, P.M., Feshbach, H.: The variational integral and the Euler equations. In: Methods of Theoretical
Physics, pp. 276-280. McGraw-Hill, New York (1953).
52. Brown ,E.S., Chan, T.F., Bresson, X.: Completely Convex Formulation of the Chan- Vese Image
Segmentation Model. International Journal of Computer Vision 98, 103–121(2012).
53. Chan, T.F., Vese, L.A.: Active contours without edges. IEEE Transactions on Image Processing 10, 266–
277 (2001).
54. Osher, S., Sethian, J.A.: Fronts propagating with curvature-dependent speed: Algorithms on Hamilton-
Jacobi formulations. Journal of Computational Physics 79, 12–49 (1988).
55. Kass, M., Witkin, A., Terzopoulos, D.: Snakes: Active contour models. International Journal of
Computer Vision 1, 321–331 (1988).
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
References Page 92
56. Andersson, T., Läthén, G., Lenz, R., Borga, M.: Modified Gradient Search for Level Set Based Image
Segmentation. IEEE Transactionson Image Processing 22, 621–630(2013).
57. Jacobs, R.A.: Increased rates of convergence through learning rate adaptation. Neural Networks 1, 295–
307 (1988).
58. . Kass, A. Witkin, and D. Terzopoulos, "Snakes: Active Contour models," Int. J. Comput. Vis vol. 1, pp.
321-331, 1988.
59. D. Peng, B. Merriman, S. Osher, H. Zhao, and M. Kang, "A PDE-based fast local level set method," J.
Comp. Phys., vol. 155, pp. 410-438, 1999.
60. A. Chander, A. Chatterjee, and P. Siarry, “A new social and momentum component adaptive PSO
algorithm for image segmentation,” Expert Systems with Applications, vol. 38, issue 5, pp. 4998-5004,
May 2011.
61. A. Chatterjee and F. Matsuno, “A geese PSO tuned fuzzy supervisor for EKF based solutions of
simultaneous localization and mapping (SLAM) problems in mobile robots,” Expert Systems with
Applications, vol. 37, issue 8, pp. 5542-5548, August 2010.
62. K. Das Sharma, A. Chatterjee, and A. Rakshit, “A random spatial lbest PSO-based hybrid strategy for
designing adaptive fuzzy controllers for a class of nonlinear systems,” IEEE Transactions on
Instrumentation and Meas Measurement, vol. 61, no. 6, pp. 1605-1612, June 2012.
63. D. Mumford and J. Shah, "Optimal approximation by piecewise smooth functions and associated
variational problems," Commun. Pure Appl. Math., vol. 42, pp. 577-685, 1989.
64. J. E. Solem, N. C. Overgaard and A. Heyden, "Initialization techniques for segmentation with the Chan
Vese model," Proceedings of the International Conference on pattern Recognition,vol. 2, pp. 171-174,
2006.
65. E. S. Brown, T. Chan and X. Bresson, "Completely Convex Formulation of the Chan-Vese Image
Segmentation Model," Int. J. Comput. Vis., vol. 98, pp. 103-121, 2012.
66. V. Caselles, F. Catté, T. Coll, and F. Dibos, "A geometric model for active contours in image
processing," Numer. Math., vol. 66, pp. 1-31, 1993.
67. S. Osher and R. Fedkiw, Level Set Methods and Dynamic Implicit Surfaces. Springer Verlag, New York,
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
References Page 93
2002.
68. T. Chan and L. Vese, "Active contours without edges," IEEE Trans. Image Process., vol. 10, no. 2, pp.
266-277, Feb. 2001.
69. S. Osher and J. A. Sethian, "Fronts propagating with curvature-dependent speed: Algorithms based on
Hamilton-Jacobi Formulation," J. Comput. Phys., vol. 79, pp. 12-49, 1988.
70. T. Chan, B. Y. Sandberg and L. Vese, "Active contours without edges for Vector-Valued Images,"
Journal of Visual Communication and Image Representation, vol. 11, pp. 130-141, 1999.
71. C. Li, C. Xu, C. Gui, and M. D. Fox, "Level set evolution without re-initialization: A new variational
formulation," in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2005, vol. 1, pp. 430-436.
72. V. Caselles, R. Kimmel, and G. Sapiro,"On geodesic active contours," Int. J. Comput. Vis vol. 22, no. 1,
pp. 61-79, 1997.
73. Shahri, B.M.A., Adeyami, I.R. and Alavi, B.M.: Intrusion Detection System Using Hybird Gsa-K-
Means. In: Proceedings of Global Engineering, Science and Technology GSA-K Means. In: Proceedings
of Global Engineering, Science and Technology Conference, 3-4 October 2013, Bay View Hotel,
Singapore, and ISBN: 978-1-922069-32-0 (2013).
74. Panda, M., Patra, M.R.: Some Clustering Algorithms to Enhance The Performance of The Network
Intrusion Detection System.Estudios de Economia Aplicada, vol. 26, no. 2: 795-801(2008).
75. Rashedi, E., Nezamabadi-pour, H., Saryazdi, S.: GSA: A Gravitational Search Algorithm. Information
Sciences, vol. 179, no. 13, Springer-Verlag 2009: 2232–2248 (2009).
76. Shahri, B.M.A, Zadeh, S.K., Adeyemi, I.R., Zainal, A.: Comparative Analysis of Gravitational Search
Algorithm and K-Means Clustering Algorithm for Intrusion Detection System. Advances in
Computational Science, Engineering and Information Technology, vol. 225, Springer International
Publishing, 2013: 307-316 (2013).
77. Naidu, V.P.S., and Raol, J.R.: Pixel-Level Image Fusion Using Wavelets and Principal Component
Analysis. Defence Science Journal, vol. 58, no. 3: 338-352 (2008).
78. Desale, R.P., Verma, S.V.: Study and Analysis Of PCA, DCT & DWT Based Image Fusion
Techniques. International Conference on Signal Processing Image Processing & Pattern Recognition
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت
References Page 94
(ICSIPR), pp. 66-69. IEEE Press, Coimbatore (2013).
79. Spearman, C.: The Proof and Measurement of Association between Two Things. International Journal of
Epidemiology, vol. 39, no, 5:1137-1150 (2010).
80. Maji, D., Biswas, M., Nath, S., Bhattacharya, S.: Robust Hierarchical Based GSA K-means
Clustering Chan-Vese Segmentation of Visible-Infrared Image Fusion Using PCA. (Accepted for
Elsevier Publication) In: Eighth international conference on image and signal processing (ICISP-
2014)
متلب سایت
MatlabSite.com
MatlabSite.com متلب سایت