ucirvine department of biomedical engineering the state of the art joerg meyer [email protected]...

UCIrvineDEPARTMENT OFBIOMEDICAL ENGINEERING

The State of the ArtThe State of the ArtThe State of the ArtThe State of the Art

Joerg MeyerJoerg [email protected]@uci.edu

April 10, 2023April 10, 2023

Joerg MeyerJoerg [email protected]@uci.edu

April 10, 2023April 10, 2023

Biomedical Image ProcessingBiomedical Image ProcessingBiomedical Image ProcessingBiomedical Image Processing

1500 ||||||||||||||||||||||||||||||||| 2003

UCIrvineBiomedical Image Processing – The State of the Art

Joerg Meyer - [email protected]

DEPARTMENT OF BIOMEDICAL ENGINEERING






X-RayX-RayX-RayX-Ray

(Eberhart’s Manual of High Frequncy Currents, Ch. 10, 1911)(Eberhart’s Manual of High Frequncy Currents, Ch. 10, 1911)(Eberhart’s Manual of High Frequncy Currents, Ch. 10, 1911)(Eberhart’s Manual of High Frequncy Currents, Ch. 10, 1911)

(Gundelach Tube, 1898 – 1905)(Gundelach Tube, 1898 – 1905)

Oak Ridge Associated UniversitiesOak Ridge Associated Universities

Health PhysicsHealth PhysicsInstrumentation MuseumInstrumentation Museum

(Gundelach Tube, 1898 – 1905)(Gundelach Tube, 1898 – 1905)

Oak Ridge Associated UniversitiesOak Ridge Associated Universities

Health PhysicsHealth PhysicsInstrumentation MuseumInstrumentation Museum





Week 0 Week 4Week 0 Week 4Week 0 Week 4Week 0 Week 4





Scaphoid BoneScaphoid BoneScaphoid BoneScaphoid Bone

Week 0 Week 4Week 0 Week 4Week 0 Week 4Week 0 Week 4





Step 1: Incision MarkingStep 1: Incision MarkingStep 1: Incision MarkingStep 1: Incision Marking





Step 2: Exposure of Fracture LineStep 2: Exposure of Fracture LineStep 2: Exposure of Fracture LineStep 2: Exposure of Fracture Line





Step 3: Screw Insertion SiteStep 3: Screw Insertion SiteStep 3: Screw Insertion SiteStep 3: Screw Insertion Site





Step 4: X-RayStep 4: X-RayStep 4: X-RayStep 4: X-Ray

Before 2 months postopBefore 2 months postop Before 2 months postopBefore 2 months postop

(Images courtesy of: Electronic Textbook of Hand Surgery, http://www.eatonhand.com)(Images courtesy of: Electronic Textbook of Hand Surgery, http://www.eatonhand.com)(Images courtesy of: Electronic Textbook of Hand Surgery, http://www.eatonhand.com)(Images courtesy of: Electronic Textbook of Hand Surgery, http://www.eatonhand.com)





High FrequencyHigh FrequencyX-Ray TubeX-Ray Tube

High FrequencyHigh FrequencyX-Ray TubeX-Ray Tube

(Eberhart’s Manual of High Frequncy Currents, Chap. 10, 1911)(Eberhart’s Manual of High Frequncy Currents, Chap. 10, 1911)(Eberhart’s Manual of High Frequncy Currents, Chap. 10, 1911)(Eberhart’s Manual of High Frequncy Currents, Chap. 10, 1911)

Portable SystemPortable SystemPortable SystemPortable System





Scaphoids Scaphoids (1mm)(1mm)

Scaphoids Scaphoids (1mm)(1mm)

MagneVu 1000MagneVu 1000(MRI)(MRI)

MagneVu 1000MagneVu 1000(MRI)(MRI)





• Problems:Problems:– TimeTime– Cost factorCost factor

• Solution:Solution:– Correlate X-ray images with 3-D modelsCorrelate X-ray images with 3-D models– Database (typical bones)Database (typical bones)

• Problems:Problems:– TimeTime– Cost factorCost factor

• Solution:Solution:– Correlate X-ray images with 3-D modelsCorrelate X-ray images with 3-D models– Database (typical bones)Database (typical bones)





Various shapes and sizes (Zimmer, 1968)Various shapes and sizes (Zimmer, 1968)Various shapes and sizes (Zimmer, 1968)Various shapes and sizes (Zimmer, 1968)

Scaphoid BonesScaphoid BonesScaphoid BonesScaphoid Bones





Classification (Herbert)Classification (Herbert)Classification (Herbert)Classification (Herbert)

Scaphoid Bone FracturesScaphoid Bone FracturesScaphoid Bone FracturesScaphoid Bone Fractures





A - C: Scaphoid view 1 - with forearm pronated 45deg. to view profile of scaphoid & STT joint

D - Scaphoid view 2 (ulnar oblique view) showing radioscaphoid joint (from Rockwood & Green)

A - C: Scaphoid view 1 - with forearm pronated 45deg. to view profile of scaphoid & STT joint

D - Scaphoid view 2 (ulnar oblique view) showing radioscaphoid joint (from Rockwood & Green)

AA BB

CC DD





• Solution:Solution:– SuperimposeSuperimpose

radiographicradiographic

scan andscan and

3-D model3-D model

– Select bestSelect best

model frommodel from

databasedatabase

• Solution:Solution:– SuperimposeSuperimpose

radiographicradiographic

scan andscan and

3-D model3-D model

– Select bestSelect best

model frommodel from

databasedatabase





• General General Question:Question:

– How to combineHow to combine

differentdifferent

modalities?modalities?

• General General Question:Question:

– How to combineHow to combine

differentdifferent

modalities?modalities?




Visible Human (CT, frozen)Visible Human (CT, frozen)Visible Human (CT, frozen)Visible Human (CT, frozen)

Joseph Paul JerniganJoseph Paul JerniganJoseph Paul JerniganJoseph Paul Jernigan




Visible Human (CT, frozen)Visible Human (CT, frozen)Visible Human (CT, frozen)Visible Human (CT, frozen)

Slice 1125Slice 1125Slice 1125Slice 1125




Visible Human (MRI)Visible Human (MRI)Visible Human (MRI)Visible Human (MRI)





Visible Human (RGB color)Visible Human (RGB color)Visible Human (RGB color)Visible Human (RGB color)





Visible Human Visible Human (3-D Reconstruction)(3-D Reconstruction)Visible Human Visible Human (3-D Reconstruction)(3-D Reconstruction)




CT ScannerCT ScannerCT ScannerCT Scanner




Image AcquisitionImage AcquisitionImage AcquisitionImage Acquisition




CT ScannerCT ScannerCT ScannerCT Scanner




Principle of a CT ScannerPrinciple of a CT ScannerPrinciple of a CT ScannerPrinciple of a CT Scanner

ObjectTranslation

Rotation

X-Ray Source

Detector

Object

X-Ray Source

Detector

a) Translation b) Rotation





Principle of an MRI ScannerPrinciple of an MRI ScannerPrinciple of an MRI ScannerPrinciple of an MRI Scanner

a) Random Spin b) directional magnetic field





a) Directional Puls (orthogonal) b) Relaxation


Principle of an MRI ScannerPrinciple of an MRI ScannerPrinciple of an MRI ScannerPrinciple of an MRI Scanner




Rhesus Monkey BrainRhesus Monkey BrainRhesus Monkey BrainRhesus Monkey Brain

• High-resolution large-scale image dataHigh-resolution large-scale image data

RGB image series (real-color, dyed), 5037 x 3871 x 1400, 76 GBRGB image series (real-color, dyed), 5037 x 3871 x 1400, 76 GB(data courtesy of Edward G. Jones, Center for Neuroscience, UC Davis)(data courtesy of Edward G. Jones, Center for Neuroscience, UC Davis)

• High-resolution large-scale image dataHigh-resolution large-scale image data

RGB image series (real-color, dyed), 5037 x 3871 x 1400, 76 GBRGB image series (real-color, dyed), 5037 x 3871 x 1400, 76 GB(data courtesy of Edward G. Jones, Center for Neuroscience, UC Davis)(data courtesy of Edward G. Jones, Center for Neuroscience, UC Davis)

• Resolution: 2666dpi Resolution: 2666dpi

Pixel spacing: 9 Pixel spacing: 9 mm

• Enables zoomingEnables zooming

down to the cell down to the cell

level.level.

• Total data size:Total data size:76 GB76 GB




3-D Reconstruction3-D Reconstruction3-D Reconstruction3-D Reconstruction




From 2-D Cross-sections to 3-DFrom 2-D Cross-sections to 3-DFrom 2-D Cross-sections to 3-DFrom 2-D Cross-sections to 3-D

RGB image series (real-color, human brain), 1472 x 1152 x 753, 3.57 GBRGB image series (real-color, human brain), 1472 x 1152 x 753, 3.57 GB(data courtesy of Arthur W. Toga, Dept. of Neurology, UCLA School of Medicine)(data courtesy of Arthur W. Toga, Dept. of Neurology, UCLA School of Medicine)

RGB image series (real-color, human brain), 1472 x 1152 x 753, 3.57 GBRGB image series (real-color, human brain), 1472 x 1152 x 753, 3.57 GB(data courtesy of Arthur W. Toga, Dept. of Neurology, UCLA School of Medicine)(data courtesy of Arthur W. Toga, Dept. of Neurology, UCLA School of Medicine)





• 3-D Volume Rendering of a Human Brain3-D Volume Rendering of a Human Brain• 3-D Volume Rendering of a Human Brain3-D Volume Rendering of a Human Brain

RGB image series (real-color, human brain), 1472 x 1152 x 753, 3.57 GB(data courtesy of Arthur W. Toga, Dept. of Neurology, UCLA School of Medicine,

image courtesy of Eric B. Lum, Ikuko Takanashi, CIPIC, UC Davis)

RGB image series (real-color, human brain), 1472 x 1152 x 753, 3.57 GB(data courtesy of Arthur W. Toga, Dept. of Neurology, UCLA School of Medicine,

image courtesy of Eric B. Lum, Ikuko Takanashi, CIPIC, UC Davis)





??





CT head (CT head (512 cross-sections, 1024 planes512 cross-sections, 1024 planes))rendered with different transparency transfer functions rendered with different transparency transfer functions




Human brainHuman brain((128 cross-sections,128 cross-sections,

220 planes220 planes))

Cancer cell Cancer cell ((256 cross-sections,256 cross-sections,

512 planes512 planes))

Ice BlockIce Block(Human brain)(Human brain)

((128 cryo-sections,128 cryo-sections,256 planes256 planes))

3-D Reconstruction3-D Reconstruction





MRI scan of a human skullMRI scan of a human skull





CT scan of a human skullCT scan of a human skull




How to scale down large-scale data?How to scale down large-scale data?How to scale down large-scale data?How to scale down large-scale data?

MM

120 CD-ROMs120 CD-ROMs 100k polygons100k polygons




Space DecompositionSpace DecompositionSpace DecompositionSpace Decomposition• Data arrangedData arranged

in slicesin slices• Data arrangedData arranged

in slicesin slices




Space DecompositionSpace DecompositionSpace DecompositionSpace Decomposition• Goal: extract subvolumeGoal: extract subvolume• Problem: data arranged in slices orProblem: data arranged in slices or

unstructured gridsunstructured grids• Necessity to touch a lot of files or dataNecessity to touch a lot of files or data• Solution: breaking up the data set into Solution: breaking up the data set into

bricksbricks• Data in bricks still too big for transmission/Data in bricks still too big for transmission/

interactive renderinginteractive rendering• Solution: multiresolution representationSolution: multiresolution representation idea: combination of octree/waveletidea: combination of octree/wavelet

• Goal: extract subvolumeGoal: extract subvolume• Problem: data arranged in slices orProblem: data arranged in slices or

unstructured gridsunstructured grids• Necessity to touch a lot of files or dataNecessity to touch a lot of files or data• Solution: breaking up the data set into Solution: breaking up the data set into

bricksbricks• Data in bricks still too big for transmission/Data in bricks still too big for transmission/

interactive renderinginteractive rendering• Solution: multiresolution representationSolution: multiresolution representation idea: combination of octree/waveletidea: combination of octree/wavelet




Space DecompositionSpace DecompositionSpace DecompositionSpace Decomposition• Data arrangedData arranged

in slicesin slices• Data arrangedData arranged

in slicesin slices




Space DecompositionSpace DecompositionSpace DecompositionSpace Decomposition• Breaking up theBreaking up the

data into bricksdata into bricks• Breaking up theBreaking up the

data into bricksdata into bricks







......







ROIROI......




Combination: Octree/WaveletCombination: Octree/WaveletCombination: Octree/WaveletCombination: Octree/Wavelet• Leaf encodingLeaf encoding• Leaf encodingLeaf encoding




Wavelet CompressionWavelet CompressionWavelet CompressionWavelet Compression

image pyramidimage pyramid

original (256 x 256)original (256 x 256) step 1step 1 step 2step 2

......





MRI scanMRI scan

original (256 x 256)original (256 x 256) n = 8n = 8

n = 2n = 2n = 3n = 3n = 4n = 4n = 5n = 5

n = 7n = 7 n = 6n = 6





Original image array

Original image array

Original VolumeOriginal Volume

L H L H

First run: x-directionFirst run: x-direction First run: x-directionFirst run: x-direction

HL HL LL LL

LH LH HL HL

HLLHLLLLLLLL

LHLLHL HLLHLL

LLH LLH

LHH LHH

HLHHLH

HHHHHH

Second run: y-directionSecond run: y-direction Third run: z-direction Third run: z-direction




Progressive ReconstructionProgressive ReconstructionProgressive ReconstructionProgressive Reconstruction

Initial stage Second level of detailInitial stage Second level of detail

Third level of detail Final reconstructed imageThird level of detail Final reconstructed image




• Cancer ResearchCancer Research

– Image processing (segmentation, classification)Image processing (segmentation, classification)

– Multi-modal imaging (CT/MRI/cryosection/confocal)Multi-modal imaging (CT/MRI/cryosection/confocal)

• NeuroscienceNeuroscience

– Rhesus Macaque Monkey Brain Atlas (NIMH)Rhesus Macaque Monkey Brain Atlas (NIMH)

– Scalable Visualization Toolkits for Bays to Brains (NPACI)Scalable Visualization Toolkits for Bays to Brains (NPACI)

• Cell PhysiologyCell Physiology

– Connectivity in Leech Giant Glial CellsConnectivity in Leech Giant Glial Cells

– Correspondence Analysis in Time-variant MicroscopicCorrespondence Analysis in Time-variant Microscopic3D Image Data3D Image Data

• Molecular DiagnosticsMolecular Diagnostics

– Genomics, Proteomics, Phylogenetic TreesGenomics, Proteomics, Phylogenetic Trees

• Cancer ResearchCancer Research

– Image processing (segmentation, classification)Image processing (segmentation, classification)

– Multi-modal imaging (CT/MRI/cryosection/confocal)Multi-modal imaging (CT/MRI/cryosection/confocal)

• NeuroscienceNeuroscience

– Rhesus Macaque Monkey Brain Atlas (NIMH)Rhesus Macaque Monkey Brain Atlas (NIMH)

– Scalable Visualization Toolkits for Bays to Brains (NPACI)Scalable Visualization Toolkits for Bays to Brains (NPACI)

• Cell PhysiologyCell Physiology

– Connectivity in Leech Giant Glial CellsConnectivity in Leech Giant Glial Cells

– Correspondence Analysis in Time-variant MicroscopicCorrespondence Analysis in Time-variant Microscopic3D Image Data3D Image Data

• Molecular DiagnosticsMolecular Diagnostics

– Genomics, Proteomics, Phylogenetic TreesGenomics, Proteomics, Phylogenetic Trees

Application AreasApplication AreasApplication AreasApplication Areas




• Projection SystemsProjection Systems– Concave Reality Projection SystemConcave Reality Projection System– RealityCenterRealityCenter– Interactive White BoardsInteractive White Boards– Active/Passive Stereo ProjectorsActive/Passive Stereo Projectors

• Spatial Tracking HardwareSpatial Tracking Hardware– Nest-of-Birds (electro-magnetic)Nest-of-Birds (electro-magnetic)– VICON Motion Tracking SystemVICON Motion Tracking System

(vision)(vision)

• Interaction HardwareInteraction Hardware– PinchGloves (data gloves)PinchGloves (data gloves)– Stylus (6dof pointing device)Stylus (6dof pointing device)

• Projection SystemsProjection Systems– Concave Reality Projection SystemConcave Reality Projection System– RealityCenterRealityCenter– Interactive White BoardsInteractive White Boards– Active/Passive Stereo ProjectorsActive/Passive Stereo Projectors

• Spatial Tracking HardwareSpatial Tracking Hardware– Nest-of-Birds (electro-magnetic)Nest-of-Birds (electro-magnetic)– VICON Motion Tracking SystemVICON Motion Tracking System

(vision)(vision)

• Interaction HardwareInteraction Hardware– PinchGloves (data gloves)PinchGloves (data gloves)– Stylus (6dof pointing device)Stylus (6dof pointing device)

InfrastructureInfrastructureInfrastructureInfrastructure




ConclusionsConclusionsConclusionsConclusions

• Engineering solutions can helpEngineering solutions can help

to make health care affordableto make health care affordable• Economic feasibilityEconomic feasibility• Data set size andData set size and

image quality increasesimage quality increases• Scalable solutionsScalable solutions




AcknowledgementsAcknowledgementsAcknowledgementsAcknowledgements• Center for Neuroscience,UC DavisCenter for Neuroscience,UC Davis

• Engineering Research Center, Mississippi State UniversityEngineering Research Center, Mississippi State University

• Center for Image Processing and Integrated ComputingCenter for Image Processing and Integrated Computing

(CIPIC), UC Davis(CIPIC), UC Davis

• The Scripps Research Institute (TSRI), La Jolla, CAThe Scripps Research Institute (TSRI), La Jolla, CA

• San Diego Supercomputer CenterSan Diego Supercomputer Center

• These projects are funded in part by the National Institute These projects are funded in part by the National Institute

of Mental Health (NIMH) through the Center for of Mental Health (NIMH) through the Center for

Neuroscience at UC Davis, and the National Partnership Neuroscience at UC Davis, and the National Partnership

for Advanced Computational Infrastructure (NPACI)for Advanced Computational Infrastructure (NPACI)

(award #10195430 00120410).(award #10195430 00120410).

• Center for Neuroscience,UC DavisCenter for Neuroscience,UC Davis

• Engineering Research Center, Mississippi State UniversityEngineering Research Center, Mississippi State University

• Center for Image Processing and Integrated ComputingCenter for Image Processing and Integrated Computing

(CIPIC), UC Davis(CIPIC), UC Davis

• The Scripps Research Institute (TSRI), La Jolla, CAThe Scripps Research Institute (TSRI), La Jolla, CA

• San Diego Supercomputer CenterSan Diego Supercomputer Center

• These projects are funded in part by the National Institute These projects are funded in part by the National Institute

of Mental Health (NIMH) through the Center for of Mental Health (NIMH) through the Center for

Neuroscience at UC Davis, and the National Partnership Neuroscience at UC Davis, and the National Partnership

for Advanced Computational Infrastructure (NPACI)for Advanced Computational Infrastructure (NPACI)

(award #10195430 00120410).(award #10195430 00120410).




That‘s all, folks!That‘s all, folks!That‘s all, folks!That‘s all, folks!

Joerg MeyerUniversity of California, IrvineBME/EECS Department644E Engineering TowerIrvine, CA 92697-2625

[email protected]://www.eng.uci.edu/~jmeyer

Questions?

ucirvine department of biomedical engineering the state of the art joerg meyer [email protected]...

Documents

ucirvine biomedical

art joerg meyer

database slide

scaphoid bones slide

mri slide

portable system slide

b b c c d d slide

database solution