![Page 1: Chapter 8 Content-Based Image Retrieval. Query By Keyword: Some textual attributes (keywords) should be maintained for each image. The image can be indexed](https://reader030.vdocument.in/reader030/viewer/2022033101/56649cf35503460f949c10db/html5/thumbnails/1.jpg)
Chapter 8 Chapter 8 Content-Based Image Content-Based Image
RetrievalRetrieval
![Page 2: Chapter 8 Content-Based Image Retrieval. Query By Keyword: Some textual attributes (keywords) should be maintained for each image. The image can be indexed](https://reader030.vdocument.in/reader030/viewer/2022033101/56649cf35503460f949c10db/html5/thumbnails/2.jpg)
Query By Keyword: Some textual attributes (keywords) should be maintained for each image. The image can be indexed according to these attributes, so that they can be rapidly retrieved when a query is issued. This type of query can be expressed in Structured Query Language (SQL).
Query By Example (QBE): User just show the system a sample image, then the system should be able to return similar images or images containing similar objects.
Image Database QueriesImage Database Queries
![Page 3: Chapter 8 Content-Based Image Retrieval. Query By Keyword: Some textual attributes (keywords) should be maintained for each image. The image can be indexed](https://reader030.vdocument.in/reader030/viewer/2022033101/56649cf35503460f949c10db/html5/thumbnails/3.jpg)
Image Distance & Image Distance & Similarity MeasuresSimilarity Measures
1. Color Similarity
2. Texture Similarity
3. Shape Similarity
4. Object & Relationship similarity
![Page 4: Chapter 8 Content-Based Image Retrieval. Query By Keyword: Some textual attributes (keywords) should be maintained for each image. The image can be indexed](https://reader030.vdocument.in/reader030/viewer/2022033101/56649cf35503460f949c10db/html5/thumbnails/4.jpg)
Color SimilarityColor Similarity
• Color percentages matching: R:20%, G:50%, B:30%
• Color histogram matching
Dhist(I,Q)=(h(I)-h(Q))TA(h(I)-h(Q))
A is a similarity matrix colors that are very similar should have similarity values close to one.
![Page 5: Chapter 8 Content-Based Image Retrieval. Query By Keyword: Some textual attributes (keywords) should be maintained for each image. The image can be indexed](https://reader030.vdocument.in/reader030/viewer/2022033101/56649cf35503460f949c10db/html5/thumbnails/5.jpg)
• Color layout matching: compares each grid square of the query to the corresponding grid square of a potential matching image and combines the results into a single image distance
where CI(g) represents the color in grid square g of a database image I and CQ(g) represents the color in the corresponding grid square g of the query image Q. some suitable representations of color are
1. Mean2. Mean and standard deviation3. Multi-bin histogram
Color SimilarityColor Similarity
g
QIcolorcolorgridded gCgCdQId ))(),((ˆ),(_
![Page 6: Chapter 8 Content-Based Image Retrieval. Query By Keyword: Some textual attributes (keywords) should be maintained for each image. The image can be indexed](https://reader030.vdocument.in/reader030/viewer/2022033101/56649cf35503460f949c10db/html5/thumbnails/6.jpg)
IQ IQ based based on on Color Color LayouLayoutt
![Page 7: Chapter 8 Content-Based Image Retrieval. Query By Keyword: Some textual attributes (keywords) should be maintained for each image. The image can be indexed](https://reader030.vdocument.in/reader030/viewer/2022033101/56649cf35503460f949c10db/html5/thumbnails/7.jpg)
• Pick and clickSuppose T(I) is a texture description vector which is a vector of numbers that summarizes the texture in a given image I (for example: Laws texture energy measures), then the texture distance measure is defined by
• Texture layout
Texture SimilarityTexture Similarity
g
QItexturetexturegridded gTgTdQId ))(),((ˆ),(_
2
__ )()(min),( QTiTQId Iiclickandpick
![Page 8: Chapter 8 Content-Based Image Retrieval. Query By Keyword: Some textual attributes (keywords) should be maintained for each image. The image can be indexed](https://reader030.vdocument.in/reader030/viewer/2022033101/56649cf35503460f949c10db/html5/thumbnails/8.jpg)
IQ IQ basebased on d on Pick Pick and and ClickClick
![Page 9: Chapter 8 Content-Based Image Retrieval. Query By Keyword: Some textual attributes (keywords) should be maintained for each image. The image can be indexed](https://reader030.vdocument.in/reader030/viewer/2022033101/56649cf35503460f949c10db/html5/thumbnails/9.jpg)
Shape SimilarityShape Similarity
1. Shape Histogram
2. Boundary Matching
3. Sketch Matching
![Page 10: Chapter 8 Content-Based Image Retrieval. Query By Keyword: Some textual attributes (keywords) should be maintained for each image. The image can be indexed](https://reader030.vdocument.in/reader030/viewer/2022033101/56649cf35503460f949c10db/html5/thumbnails/10.jpg)
1. 1. Shape HistogramShape Histogram• Projection matchingProjection matching
• Horizontal & vertical projection: Each row and each column become a bin in the histogram. The count that is stored in a bin is the number of 1-pixels that appear in that row or column.• Diagonal projection: An alternative is to define the bins from the top left to the bottom right of the shape.
• Size invariant the number of row bins and the number of column bins in the bounding box can be fixed, histograms can be normalized before matching.• Translation invariant • Rotation invariant compute the axis of the best-fitting ellipse and rotate the shape
![Page 11: Chapter 8 Content-Based Image Retrieval. Query By Keyword: Some textual attributes (keywords) should be maintained for each image. The image can be indexed](https://reader030.vdocument.in/reader030/viewer/2022033101/56649cf35503460f949c10db/html5/thumbnails/11.jpg)
Horizontal and vertical Horizontal and vertical projectionsprojections
![Page 12: Chapter 8 Content-Based Image Retrieval. Query By Keyword: Some textual attributes (keywords) should be maintained for each image. The image can be indexed](https://reader030.vdocument.in/reader030/viewer/2022033101/56649cf35503460f949c10db/html5/thumbnails/12.jpg)
Diagonal projectionDiagonal projection
![Page 13: Chapter 8 Content-Based Image Retrieval. Query By Keyword: Some textual attributes (keywords) should be maintained for each image. The image can be indexed](https://reader030.vdocument.in/reader030/viewer/2022033101/56649cf35503460f949c10db/html5/thumbnails/13.jpg)
• Orientation histogramOrientation histogram
• Construct a histogram over the tangent angle at each pixel on the boundary of the shape.
• Size invariant histograms can be normalized before matching.• Translation invariant• Rotation invariant choosing the bin with the largest count to be the first bin.• Starting point invariant
1. 1. Shape HistogramShape Histogram
![Page 14: Chapter 8 Content-Based Image Retrieval. Query By Keyword: Some textual attributes (keywords) should be maintained for each image. The image can be indexed](https://reader030.vdocument.in/reader030/viewer/2022033101/56649cf35503460f949c10db/html5/thumbnails/14.jpg)
2. 2. Boundary MatchingBoundary Matching
11D Fourier TransformD Fourier Transform on the boundary
1
0
2
1
0
2
10,
10,
N
n
N
knj
kn
N
k
N
knj
n
k
nnn
Nneau
NkN
eua
jyxu
![Page 15: Chapter 8 Content-Based Image Retrieval. Query By Keyword: Some textual attributes (keywords) should be maintained for each image. The image can be indexed](https://reader030.vdocument.in/reader030/viewer/2022033101/56649cf35503460f949c10db/html5/thumbnails/15.jpg)
• If only the first M coefficients (a0, a1, …, aM-1) are used, then
is an approximation of un
• the coefficients (a0, a1, …, aM-1) is called Fourier DescriptorsFourier Descriptors
• The Fourier distance measure is defined as:
1
0
2
10,ˆM
n
N
knj
kn Nneau
1
0
2),(
M
n
Qn
InFourier aaQId
Fourier DescriptorsFourier Descriptors
![Page 16: Chapter 8 Content-Based Image Retrieval. Query By Keyword: Some textual attributes (keywords) should be maintained for each image. The image can be indexed](https://reader030.vdocument.in/reader030/viewer/2022033101/56649cf35503460f949c10db/html5/thumbnails/16.jpg)
Properties of Fourier Properties of Fourier DescriptorsDescriptors
Simple geometric transformations of a boundary, such as translation, rotation, and scaling, are related to simple operations of the boundary’s Fourier descriptors.
![Page 17: Chapter 8 Content-Based Image Retrieval. Query By Keyword: Some textual attributes (keywords) should be maintained for each image. The image can be indexed](https://reader030.vdocument.in/reader030/viewer/2022033101/56649cf35503460f949c10db/html5/thumbnails/17.jpg)
11 constant M-k,a
aQk
Ik
A formula for Fourier Descriptor that is invariant to translation, scaling, rotation, and starting point.
A secret formulaA secret formula
![Page 18: Chapter 8 Content-Based Image Retrieval. Query By Keyword: Some textual attributes (keywords) should be maintained for each image. The image can be indexed](https://reader030.vdocument.in/reader030/viewer/2022033101/56649cf35503460f949c10db/html5/thumbnails/18.jpg)
IQ based IQ based on on Boundary Boundary MatchingMatching
![Page 19: Chapter 8 Content-Based Image Retrieval. Query By Keyword: Some textual attributes (keywords) should be maintained for each image. The image can be indexed](https://reader030.vdocument.in/reader030/viewer/2022033101/56649cf35503460f949c10db/html5/thumbnails/19.jpg)
3. 3. Sketch MatchingSketch Matching
1. Affine transformation to specified size and applying median filter.
2. Edge detection using a gradient-based edge-finding algorithm Refined edge image
3. Thinning and shrinking Abstract image
4. The images are divided into grid squares and matching is performed based on local correlation.
![Page 20: Chapter 8 Content-Based Image Retrieval. Query By Keyword: Some textual attributes (keywords) should be maintained for each image. The image can be indexed](https://reader030.vdocument.in/reader030/viewer/2022033101/56649cf35503460f949c10db/html5/thumbnails/20.jpg)
The sketch distance measure is the inverse of the sum of each of the local correlations
where I(g) refres to grid square g of the abstract image I, Q(g) refers to grid square g of the linear sketch resulting from query image Q.
g nncorrelation
sketchgQgIshiftd
QId)(),(ˆmax
1),(
3. 3. Sketch MatchingSketch Matching
![Page 21: Chapter 8 Content-Based Image Retrieval. Query By Keyword: Some textual attributes (keywords) should be maintained for each image. The image can be indexed](https://reader030.vdocument.in/reader030/viewer/2022033101/56649cf35503460f949c10db/html5/thumbnails/21.jpg)
Object and Relational Object and Relational SimilaritySimilarity
• Face finding: Neural net classifier
• Flesh finding:
then threshold based on
22
1
12
),(tan
)(
2
)()()(
)()(
)(
yg
yg
y
g
BRsaturation
BRhue
ImedImedtexture
RLGLBLB
GLRLR
GLI