![Page 1: Dr. Yu(Jade) Qian y.qian@mdx.ac.uk y.qian@mdx.ac.uk](https://reader031.vdocument.in/reader031/viewer/2022032312/56649dea5503460f94ae5434/html5/thumbnails/1.jpg)
Dr. Yu(Jade) Qian [email protected]
![Page 2: Dr. Yu(Jade) Qian y.qian@mdx.ac.uk y.qian@mdx.ac.uk](https://reader031.vdocument.in/reader031/viewer/2022032312/56649dea5503460f94ae5434/html5/thumbnails/2.jpg)
Content
• Introduction of MIRAGE project• Content-based 3D brain images retrieval and
visualization• Conclusion and future work• Demonstration
![Page 4: Dr. Yu(Jade) Qian y.qian@mdx.ac.uk y.qian@mdx.ac.uk](https://reader031.vdocument.in/reader031/viewer/2022032312/56649dea5503460f94ae5434/html5/thumbnails/4.jpg)
MIRAGE(Middlesex medical Image Repository with a CBIR ArchivinG Environment)
Phase 1: MIRAGE- from Creation to Archiving • Strand : Start-up Repository funded by JISC• Between Apr. 2009- Sep. 2010.• Aim: To develop a repository of medical images benefiting MSc and research students in
the immediate term and serve a wider community in the long term in providing a rich supply of medical images for data mining, to complement MU current online e-learning system OASIS+.
Phase 2: MIRAGE 2011 – from Archiving to Creation• Strand : Take-up and Embedding funded by JISC• Between Feb. 2011- Oct. 2011.• Aim: To enrich the current repository MIRAGE with two necessities of ‘3D Viewer’ and
‘Uploading’ to meet users’ needs, leading to a sustainable, usable and flexible model of data management.
![Page 5: Dr. Yu(Jade) Qian y.qian@mdx.ac.uk y.qian@mdx.ac.uk](https://reader031.vdocument.in/reader031/viewer/2022032312/56649dea5503460f94ae5434/html5/thumbnails/5.jpg)
Framework for MRIAGE I &II
![Page 6: Dr. Yu(Jade) Qian y.qian@mdx.ac.uk y.qian@mdx.ac.uk](https://reader031.vdocument.in/reader031/viewer/2022032312/56649dea5503460f94ae5434/html5/thumbnails/6.jpg)
Phase 1: MIRAGE – Online System(1)
• Server side: Image collection: Accommodating 100,000 2D images and 100 3D
images Visual feature extraction: Pre-processing off-line using C++ and Perl. Indexing file creation
• Client side: Interface based on PHP generate dynamic web pages.
• Client – Server communication protocol: MRML: a XML based protocol
![Page 7: Dr. Yu(Jade) Qian y.qian@mdx.ac.uk y.qian@mdx.ac.uk](https://reader031.vdocument.in/reader031/viewer/2022032312/56649dea5503460f94ae5434/html5/thumbnails/7.jpg)
Phase 1: MIRAGE – Online System(2)Interface
(1) Home page (2) Query and retrieval results
![Page 8: Dr. Yu(Jade) Qian y.qian@mdx.ac.uk y.qian@mdx.ac.uk](https://reader031.vdocument.in/reader031/viewer/2022032312/56649dea5503460f94ae5434/html5/thumbnails/8.jpg)
Phase 2: MIRAGE2011 – Online System(1)Image Uploading
MIRAGE MIRAGE2011
![Page 9: Dr. Yu(Jade) Qian y.qian@mdx.ac.uk y.qian@mdx.ac.uk](https://reader031.vdocument.in/reader031/viewer/2022032312/56649dea5503460f94ae5434/html5/thumbnails/9.jpg)
Phase 2: MIRAGE2011 – Online System(2)3D Viewer
MontageMIRAGE MIRAGE 2011
![Page 10: Dr. Yu(Jade) Qian y.qian@mdx.ac.uk y.qian@mdx.ac.uk](https://reader031.vdocument.in/reader031/viewer/2022032312/56649dea5503460f94ae5434/html5/thumbnails/10.jpg)
PART II
Content-based 3D Brain Images Retrieval and Visualization
Y. Qian, X. Gao , M. Loomes, R. Comley, B. Barn, R. Hui, Z. Tian, Content-based Retrieval of 3D Medical Images, eTELEMED 2011, February, 2011.
(Best paper award, has been invited to be extended to a journal paper).
![Page 11: Dr. Yu(Jade) Qian y.qian@mdx.ac.uk y.qian@mdx.ac.uk](https://reader031.vdocument.in/reader031/viewer/2022032312/56649dea5503460f94ae5434/html5/thumbnails/11.jpg)
2D brain images ----- 3D Brain
Shape-based Surface of a 3D object(e.g. tumor) Texture-based Inside of a 3D object( e.g.textures representing tissue structure properties)
Aim: To develop a fast texture-based 3D brain retrieval method
CBIR for 3D Brain Image ------Introduction
![Page 12: Dr. Yu(Jade) Qian y.qian@mdx.ac.uk y.qian@mdx.ac.uk](https://reader031.vdocument.in/reader031/viewer/2022032312/56649dea5503460f94ae5434/html5/thumbnails/12.jpg)
CBIR for 3D Brain Image ---Methodology(1)Proposed Framework
![Page 13: Dr. Yu(Jade) Qian y.qian@mdx.ac.uk y.qian@mdx.ac.uk](https://reader031.vdocument.in/reader031/viewer/2022032312/56649dea5503460f94ae5434/html5/thumbnails/13.jpg)
CBIR for 3D Brain Image ---Methodology(2)Pre-processing
1) Spatial Normalization---Statistical Parametric Mapping (SPM5) Transform each individual brain into a standard brain template
2) Divide 3D brain into 64 non-overlapping equally sized blocks
![Page 14: Dr. Yu(Jade) Qian y.qian@mdx.ac.uk y.qian@mdx.ac.uk](https://reader031.vdocument.in/reader031/viewer/2022032312/56649dea5503460f94ae5434/html5/thumbnails/14.jpg)
CBIR for 3D Brain Image ---Methodology(3)Extraction of Volumetric Texture Features
1) 3D Grey Level Co-occurrence Matrices (3D GLCM)
2) 3D Wavelet Transform (3D WT)3) 3D Gabor Transform (3D GT)4) 3D Local Binary Pattern (3D LBP)
![Page 15: Dr. Yu(Jade) Qian y.qian@mdx.ac.uk y.qian@mdx.ac.uk](https://reader031.vdocument.in/reader031/viewer/2022032312/56649dea5503460f94ae5434/html5/thumbnails/15.jpg)
Extraction of Volumetric Textures (1) ------ 3D Grey Level Co-occurrence Matrices (3D GLCM)
3D GLCM is two dimensional matrices of the joint probability p(i,j) of occurrence of a pair of gray values (i,j) separated by a displacement d = (dx,dy,dz).
Formula:
Feature: 52 Displacement vectors: 4 distance * 13 direction = 52 4 Haralick texture features: energy, entropy, contrast and homogeneity Feature vector: 208 components (=4 (features) * 52 (matrices)).
13 directions
![Page 16: Dr. Yu(Jade) Qian y.qian@mdx.ac.uk y.qian@mdx.ac.uk](https://reader031.vdocument.in/reader031/viewer/2022032312/56649dea5503460f94ae5434/html5/thumbnails/16.jpg)
Extraction of Volumetric Textures (2) ------ 3D Wavelet Transform (3D WT)
3D WT provides a spatial and frequency representation of a volumetric image.
Two scale 3D Wavelet Transform:
Feature:
Mean and Standard deviation Feature vector: 30 components (2 (features) +15 (sub-bands))
![Page 17: Dr. Yu(Jade) Qian y.qian@mdx.ac.uk y.qian@mdx.ac.uk](https://reader031.vdocument.in/reader031/viewer/2022032312/56649dea5503460f94ae5434/html5/thumbnails/17.jpg)
Extraction of Volumetric Textures (3) ------ 3D Gabor Transform (3D GT)
A set of 3D Gabor filters:
Gabor Transform:
Feature: 144 Gabor filters 4 (F) *6(θ)*6(Φ) =144 Mean and Standard deviation Feature vector: 288 components (2 (features) +144(filters))
zFyFxFjzyxgFzyxg cossinsincossin2exp,,,,,,,^
iiii FzyxgzyxfGT ,,,,,*,,144...3,2,1i
![Page 18: Dr. Yu(Jade) Qian y.qian@mdx.ac.uk y.qian@mdx.ac.uk](https://reader031.vdocument.in/reader031/viewer/2022032312/56649dea5503460f94ae5434/html5/thumbnails/18.jpg)
Extraction of Volumetric Textures (4) ------ 3D Local Binary Pattern (3D LBP)
Local binary pattern(LBP) is a set of binary code Ci to define texture in a local neighborhood (p,r). A histogram Hi is then generated to calculate the occurrences of different binary patterns.
LBP on three orthogonal planes (LBP-TOP), i.e., XY, XZ, and YZ planes, expressed as
Feature: 59 binary patterns Feature vector: 177 components (=59(patterns)*3(planes)
![Page 19: Dr. Yu(Jade) Qian y.qian@mdx.ac.uk y.qian@mdx.ac.uk](https://reader031.vdocument.in/reader031/viewer/2022032312/56649dea5503460f94ae5434/html5/thumbnails/19.jpg)
CBIR for 3D Brain Image ---Methodology(4)Retrieval ---Similarity Measurement
Histogram Intersection(3D LBP)
Normalized Euclidean distance (3D GLCM,3D WT,3D GT)
i
ii IQIQD ,min,
2
,
i i
ii IQIQD
![Page 20: Dr. Yu(Jade) Qian y.qian@mdx.ac.uk y.qian@mdx.ac.uk](https://reader031.vdocument.in/reader031/viewer/2022032312/56649dea5503460f94ae5434/html5/thumbnails/20.jpg)
CBIR for 3D Brain Image ---Methodology(5) Lesion Detection
Assume bilateral symmetry of a normal brain along its mid-plane
![Page 21: Dr. Yu(Jade) Qian y.qian@mdx.ac.uk y.qian@mdx.ac.uk](https://reader031.vdocument.in/reader031/viewer/2022032312/56649dea5503460f94ae5434/html5/thumbnails/21.jpg)
Evaluation ---- Test Dataset
100 MR brain images
Size: 256 256 44
DICOM (Digital Imaging and Communications in Medicine)
format
Collected from Neuro-imaging Centre at Beijing General
Navy Hospital, China
![Page 22: Dr. Yu(Jade) Qian y.qian@mdx.ac.uk y.qian@mdx.ac.uk](https://reader031.vdocument.in/reader031/viewer/2022032312/56649dea5503460f94ae5434/html5/thumbnails/22.jpg)
Experimental Results(1) ------ Lesion Detection
![Page 23: Dr. Yu(Jade) Qian y.qian@mdx.ac.uk y.qian@mdx.ac.uk](https://reader031.vdocument.in/reader031/viewer/2022032312/56649dea5503460f94ae5434/html5/thumbnails/23.jpg)
Experimental Results(2) -------Retrieval
Comparative results demonstrate that LBP outperforms four 3D texture methods in terms of retrieval precision and processing speed.
![Page 24: Dr. Yu(Jade) Qian y.qian@mdx.ac.uk y.qian@mdx.ac.uk](https://reader031.vdocument.in/reader031/viewer/2022032312/56649dea5503460f94ae5434/html5/thumbnails/24.jpg)
Experimental Results(3) -------Query time
The query time with VOI selection offers 4 times faster operation than that without.
![Page 25: Dr. Yu(Jade) Qian y.qian@mdx.ac.uk y.qian@mdx.ac.uk](https://reader031.vdocument.in/reader031/viewer/2022032312/56649dea5503460f94ae5434/html5/thumbnails/25.jpg)
3D Brain Visualization(1)
![Page 26: Dr. Yu(Jade) Qian y.qian@mdx.ac.uk y.qian@mdx.ac.uk](https://reader031.vdocument.in/reader031/viewer/2022032312/56649dea5503460f94ae5434/html5/thumbnails/26.jpg)
3D Brain Visualization(2)
![Page 27: Dr. Yu(Jade) Qian y.qian@mdx.ac.uk y.qian@mdx.ac.uk](https://reader031.vdocument.in/reader031/viewer/2022032312/56649dea5503460f94ae5434/html5/thumbnails/27.jpg)
CBIR for 3D Brain Image------ On-line system(1):
![Page 28: Dr. Yu(Jade) Qian y.qian@mdx.ac.uk y.qian@mdx.ac.uk](https://reader031.vdocument.in/reader031/viewer/2022032312/56649dea5503460f94ae5434/html5/thumbnails/28.jpg)
CBIR for 3D Brain Image------ On-line system(2):
• Server side: 100 3D brain images(DICOM format to JPG format) 3D visual feature extraction(4 methods): Off-line pre-
processing using Matlab. 3D visualization: using Matlab
• Client side: Interface based on PHP generate dynamic web pages.
![Page 29: Dr. Yu(Jade) Qian y.qian@mdx.ac.uk y.qian@mdx.ac.uk](https://reader031.vdocument.in/reader031/viewer/2022032312/56649dea5503460f94ae5434/html5/thumbnails/29.jpg)
PART III
Conclusion and Future Work
![Page 30: Dr. Yu(Jade) Qian y.qian@mdx.ac.uk y.qian@mdx.ac.uk](https://reader031.vdocument.in/reader031/viewer/2022032312/56649dea5503460f94ae5434/html5/thumbnails/30.jpg)
Conclusion for MIRAGE (Middlesex medical Image Repository with a CBIR ArchivinG Environment)
• Create Middlesex medical Image repository ( ~100000 2D images and 100 3D brain images)
• Create CBIR archiving environment for 2D and 3D medical images.
![Page 31: Dr. Yu(Jade) Qian y.qian@mdx.ac.uk y.qian@mdx.ac.uk](https://reader031.vdocument.in/reader031/viewer/2022032312/56649dea5503460f94ae5434/html5/thumbnails/31.jpg)
Future Work(1)------Continue working on 3D Brain Image
• Test on the larger dataset and enrich our repository
• Research on clinical purpose (EC FP7)• ------ Collaborate with Neuro-imaging Centre at Beijing
General Navy Hospital, China.
![Page 32: Dr. Yu(Jade) Qian y.qian@mdx.ac.uk y.qian@mdx.ac.uk](https://reader031.vdocument.in/reader031/viewer/2022032312/56649dea5503460f94ae5434/html5/thumbnails/32.jpg)
UltrasonixTABLET Ultrasound scanner
Future Work(2)------Echocardiogram Video Clip
B-mode 2D Video Clip
B-mode and M-mode Video clip
Colour Doppler Video Clip
Enrich our repositoryResearch for clinical purpose(EC FP7) ------ Collaborate with First Hospital of Tsinghua University, China.
![Page 33: Dr. Yu(Jade) Qian y.qian@mdx.ac.uk y.qian@mdx.ac.uk](https://reader031.vdocument.in/reader031/viewer/2022032312/56649dea5503460f94ae5434/html5/thumbnails/33.jpg)
MDX Grid Machine
Future Work(3) ------- Grid Computing
![Page 35: Dr. Yu(Jade) Qian y.qian@mdx.ac.uk y.qian@mdx.ac.uk](https://reader031.vdocument.in/reader031/viewer/2022032312/56649dea5503460f94ae5434/html5/thumbnails/35.jpg)