ucirvine department of biomedical engineering the state of the art joerg meyer jmeyer@uci.edu...

UCIrvineDEPARTMENT OFBIOMEDICAL ENGINEERING

The State of the ArtThe State of the ArtThe State of the ArtThe State of the Art

Joerg MeyerJoerg Meyerjmeyer@uci.edujmeyer@uci.edu

April 10, 2023April 10, 2023

Joerg MeyerJoerg Meyerjmeyer@uci.edujmeyer@uci.edu

April 10, 2023April 10, 2023

Biomedical Image ProcessingBiomedical Image ProcessingBiomedical Image ProcessingBiomedical Image Processing

1500 ||||||||||||||||||||||||||||||||| 2003

UCIrvineBiomedical Image Processing – The State of the Art

Joerg Meyer - jmeyer@uci.edu

DEPARTMENT OF BIOMEDICAL ENGINEERING

X-RayX-RayX-RayX-Ray

(Eberhart’s Manual of High Frequncy Currents, Ch. 10, 1911)(Eberhart’s Manual of High Frequncy Currents, Ch. 10, 1911)(Eberhart’s Manual of High Frequncy Currents, Ch. 10, 1911)(Eberhart’s Manual of High Frequncy Currents, Ch. 10, 1911)

(Gundelach Tube, 1898 – 1905)(Gundelach Tube, 1898 – 1905)

Oak Ridge Associated UniversitiesOak Ridge Associated Universities

Health PhysicsHealth PhysicsInstrumentation MuseumInstrumentation Museum

(Gundelach Tube, 1898 – 1905)(Gundelach Tube, 1898 – 1905)

Oak Ridge Associated UniversitiesOak Ridge Associated Universities

Health PhysicsHealth PhysicsInstrumentation MuseumInstrumentation Museum

Week 0 Week 4Week 0 Week 4Week 0 Week 4Week 0 Week 4

Scaphoid BoneScaphoid BoneScaphoid BoneScaphoid Bone

Week 0 Week 4Week 0 Week 4Week 0 Week 4Week 0 Week 4

Step 1: Incision MarkingStep 1: Incision MarkingStep 1: Incision MarkingStep 1: Incision Marking

Step 2: Exposure of Fracture LineStep 2: Exposure of Fracture LineStep 2: Exposure of Fracture LineStep 2: Exposure of Fracture Line

Step 3: Screw Insertion SiteStep 3: Screw Insertion SiteStep 3: Screw Insertion SiteStep 3: Screw Insertion Site

Step 4: X-RayStep 4: X-RayStep 4: X-RayStep 4: X-Ray

Before 2 months postopBefore 2 months postop Before 2 months postopBefore 2 months postop

(Images courtesy of: Electronic Textbook of Hand Surgery, http://www.eatonhand.com)(Images courtesy of: Electronic Textbook of Hand Surgery, http://www.eatonhand.com)(Images courtesy of: Electronic Textbook of Hand Surgery, http://www.eatonhand.com)(Images courtesy of: Electronic Textbook of Hand Surgery, http://www.eatonhand.com)

High FrequencyHigh FrequencyX-Ray TubeX-Ray Tube

(Eberhart’s Manual of High Frequncy Currents, Chap. 10, 1911)(Eberhart’s Manual of High Frequncy Currents, Chap. 10, 1911)(Eberhart’s Manual of High Frequncy Currents, Chap. 10, 1911)(Eberhart’s Manual of High Frequncy Currents, Chap. 10, 1911)

Portable SystemPortable SystemPortable SystemPortable System

Scaphoids Scaphoids (1mm)(1mm)

MagneVu 1000MagneVu 1000(MRI)(MRI)

• Problems:Problems:– TimeTime– Cost factorCost factor

• Solution:Solution:– Correlate X-ray images with 3-D modelsCorrelate X-ray images with 3-D models– Database (typical bones)Database (typical bones)

• Problems:Problems:– TimeTime– Cost factorCost factor

• Solution:Solution:– Correlate X-ray images with 3-D modelsCorrelate X-ray images with 3-D models– Database (typical bones)Database (typical bones)

Various shapes and sizes (Zimmer, 1968)Various shapes and sizes (Zimmer, 1968)Various shapes and sizes (Zimmer, 1968)Various shapes and sizes (Zimmer, 1968)

Scaphoid BonesScaphoid BonesScaphoid BonesScaphoid Bones

Classification (Herbert)Classification (Herbert)Classification (Herbert)Classification (Herbert)

Scaphoid Bone FracturesScaphoid Bone FracturesScaphoid Bone FracturesScaphoid Bone Fractures

A - C: Scaphoid view 1 - with forearm pronated 45deg. to view profile of scaphoid & STT joint

D - Scaphoid view 2 (ulnar oblique view) showing radioscaphoid joint (from Rockwood & Green)

A - C: Scaphoid view 1 - with forearm pronated 45deg. to view profile of scaphoid & STT joint

D - Scaphoid view 2 (ulnar oblique view) showing radioscaphoid joint (from Rockwood & Green)

• Solution:Solution:– SuperimposeSuperimpose

radiographicradiographic

scan andscan and

3-D model3-D model

– Select bestSelect best

model frommodel from

databasedatabase

• Solution:Solution:– SuperimposeSuperimpose

radiographicradiographic

scan andscan and

3-D model3-D model

– Select bestSelect best

model frommodel from

databasedatabase

• General General Question:Question:

– How to combineHow to combine

differentdifferent

modalities?modalities?

• General General Question:Question:

– How to combineHow to combine

differentdifferent

modalities?modalities?

Visible Human (CT, frozen)Visible Human (CT, frozen)Visible Human (CT, frozen)Visible Human (CT, frozen)

Joseph Paul JerniganJoseph Paul JerniganJoseph Paul JerniganJoseph Paul Jernigan

Visible Human (CT, frozen)Visible Human (CT, frozen)Visible Human (CT, frozen)Visible Human (CT, frozen)

Slice 1125Slice 1125Slice 1125Slice 1125

Visible Human (MRI)Visible Human (MRI)Visible Human (MRI)Visible Human (MRI)

Visible Human (RGB color)Visible Human (RGB color)Visible Human (RGB color)Visible Human (RGB color)

Visible Human Visible Human (3-D Reconstruction)(3-D Reconstruction)Visible Human Visible Human (3-D Reconstruction)(3-D Reconstruction)

CT ScannerCT ScannerCT ScannerCT Scanner

Image AcquisitionImage AcquisitionImage AcquisitionImage Acquisition

Principle of a CT ScannerPrinciple of a CT ScannerPrinciple of a CT ScannerPrinciple of a CT Scanner

ObjectTranslation

Rotation

X-Ray Source

Detector

Object

X-Ray Source

Detector

a) Translation b) Rotation

Principle of an MRI ScannerPrinciple of an MRI ScannerPrinciple of an MRI ScannerPrinciple of an MRI Scanner

a) Random Spin b) directional magnetic field

a) Directional Puls (orthogonal) b) Relaxation

Principle of an MRI ScannerPrinciple of an MRI ScannerPrinciple of an MRI ScannerPrinciple of an MRI Scanner

Rhesus Monkey BrainRhesus Monkey BrainRhesus Monkey BrainRhesus Monkey Brain

• High-resolution large-scale image dataHigh-resolution large-scale image data

RGB image series (real-color, dyed), 5037 x 3871 x 1400, 76 GBRGB image series (real-color, dyed), 5037 x 3871 x 1400, 76 GB(data courtesy of Edward G. Jones, Center for Neuroscience, UC Davis)(data courtesy of Edward G. Jones, Center for Neuroscience, UC Davis)

• High-resolution large-scale image dataHigh-resolution large-scale image data

RGB image series (real-color, dyed), 5037 x 3871 x 1400, 76 GBRGB image series (real-color, dyed), 5037 x 3871 x 1400, 76 GB(data courtesy of Edward G. Jones, Center for Neuroscience, UC Davis)(data courtesy of Edward G. Jones, Center for Neuroscience, UC Davis)

• Resolution: 2666dpi Resolution: 2666dpi

Pixel spacing: 9 Pixel spacing: 9 mm

• Enables zoomingEnables zooming

down to the cell down to the cell

level.level.

• Total data size:Total data size:76 GB76 GB

3-D Reconstruction3-D Reconstruction3-D Reconstruction3-D Reconstruction

From 2-D Cross-sections to 3-DFrom 2-D Cross-sections to 3-DFrom 2-D Cross-sections to 3-DFrom 2-D Cross-sections to 3-D

RGB image series (real-color, human brain), 1472 x 1152 x 753, 3.57 GBRGB image series (real-color, human brain), 1472 x 1152 x 753, 3.57 GB(data courtesy of Arthur W. Toga, Dept. of Neurology, UCLA School of Medicine)(data courtesy of Arthur W. Toga, Dept. of Neurology, UCLA School of Medicine)

• 3-D Volume Rendering of a Human Brain3-D Volume Rendering of a Human Brain• 3-D Volume Rendering of a Human Brain3-D Volume Rendering of a Human Brain

RGB image series (real-color, human brain), 1472 x 1152 x 753, 3.57 GB(data courtesy of Arthur W. Toga, Dept. of Neurology, UCLA School of Medicine,

image courtesy of Eric B. Lum, Ikuko Takanashi, CIPIC, UC Davis)

RGB image series (real-color, human brain), 1472 x 1152 x 753, 3.57 GB(data courtesy of Arthur W. Toga, Dept. of Neurology, UCLA School of Medicine,

image courtesy of Eric B. Lum, Ikuko Takanashi, CIPIC, UC Davis)

CT head (CT head (512 cross-sections, 1024 planes512 cross-sections, 1024 planes))rendered with different transparency transfer functions rendered with different transparency transfer functions

Human brainHuman brain((128 cross-sections,128 cross-sections,

220 planes220 planes))

Cancer cell Cancer cell ((256 cross-sections,256 cross-sections,

512 planes512 planes))

Ice BlockIce Block(Human brain)(Human brain)

((128 cryo-sections,128 cryo-sections,256 planes256 planes))

3-D Reconstruction3-D Reconstruction

MRI scan of a human skullMRI scan of a human skull

CT scan of a human skullCT scan of a human skull

How to scale down large-scale data?How to scale down large-scale data?How to scale down large-scale data?How to scale down large-scale data?

120 CD-ROMs120 CD-ROMs 100k polygons100k polygons

Space DecompositionSpace DecompositionSpace DecompositionSpace Decomposition• Data arrangedData arranged

in slicesin slices• Data arrangedData arranged

in slicesin slices

Space DecompositionSpace DecompositionSpace DecompositionSpace Decomposition• Goal: extract subvolumeGoal: extract subvolume• Problem: data arranged in slices orProblem: data arranged in slices or

unstructured gridsunstructured grids• Necessity to touch a lot of files or dataNecessity to touch a lot of files or data• Solution: breaking up the data set into Solution: breaking up the data set into

bricksbricks• Data in bricks still too big for transmission/Data in bricks still too big for transmission/

interactive renderinginteractive rendering• Solution: multiresolution representationSolution: multiresolution representation idea: combination of octree/waveletidea: combination of octree/wavelet

• Goal: extract subvolumeGoal: extract subvolume• Problem: data arranged in slices orProblem: data arranged in slices or

unstructured gridsunstructured grids• Necessity to touch a lot of files or dataNecessity to touch a lot of files or data• Solution: breaking up the data set into Solution: breaking up the data set into

bricksbricks• Data in bricks still too big for transmission/Data in bricks still too big for transmission/

interactive renderinginteractive rendering• Solution: multiresolution representationSolution: multiresolution representation idea: combination of octree/waveletidea: combination of octree/wavelet

in slicesin slices

Space DecompositionSpace DecompositionSpace DecompositionSpace Decomposition• Breaking up theBreaking up the

data into bricksdata into bricks• Breaking up theBreaking up the

data into bricksdata into bricks

......

ROIROI......

Combination: Octree/WaveletCombination: Octree/WaveletCombination: Octree/WaveletCombination: Octree/Wavelet• Leaf encodingLeaf encoding• Leaf encodingLeaf encoding

Wavelet CompressionWavelet CompressionWavelet CompressionWavelet Compression

image pyramidimage pyramid

original (256 x 256)original (256 x 256) step 1step 1 step 2step 2

......

MRI scanMRI scan

original (256 x 256)original (256 x 256) n = 8n = 8

n = 2n = 2n = 3n = 3n = 4n = 4n = 5n = 5

n = 7n = 7 n = 6n = 6

Original image array

Original VolumeOriginal Volume

L H L H

First run: x-directionFirst run: x-direction First run: x-directionFirst run: x-direction

HL HL LL LL

LH LH HL HL

HLLHLLLLLLLL

LHLLHL HLLHLL

LLH LLH

LHH LHH

HLHHLH

HHHHHH

Second run: y-directionSecond run: y-direction Third run: z-direction Third run: z-direction

Progressive ReconstructionProgressive ReconstructionProgressive ReconstructionProgressive Reconstruction

Initial stage Second level of detailInitial stage Second level of detail

Third level of detail Final reconstructed imageThird level of detail Final reconstructed image

• Cancer ResearchCancer Research

– Image processing (segmentation, classification)Image processing (segmentation, classification)

– Multi-modal imaging (CT/MRI/cryosection/confocal)Multi-modal imaging (CT/MRI/cryosection/confocal)

• NeuroscienceNeuroscience

– Rhesus Macaque Monkey Brain Atlas (NIMH)Rhesus Macaque Monkey Brain Atlas (NIMH)

– Scalable Visualization Toolkits for Bays to Brains (NPACI)Scalable Visualization Toolkits for Bays to Brains (NPACI)

• Cell PhysiologyCell Physiology

– Connectivity in Leech Giant Glial CellsConnectivity in Leech Giant Glial Cells

– Correspondence Analysis in Time-variant MicroscopicCorrespondence Analysis in Time-variant Microscopic3D Image Data3D Image Data

• Molecular DiagnosticsMolecular Diagnostics

– Genomics, Proteomics, Phylogenetic TreesGenomics, Proteomics, Phylogenetic Trees

• Cancer ResearchCancer Research

– Image processing (segmentation, classification)Image processing (segmentation, classification)

– Multi-modal imaging (CT/MRI/cryosection/confocal)Multi-modal imaging (CT/MRI/cryosection/confocal)

• NeuroscienceNeuroscience

– Rhesus Macaque Monkey Brain Atlas (NIMH)Rhesus Macaque Monkey Brain Atlas (NIMH)

– Scalable Visualization Toolkits for Bays to Brains (NPACI)Scalable Visualization Toolkits for Bays to Brains (NPACI)

• Cell PhysiologyCell Physiology

– Connectivity in Leech Giant Glial CellsConnectivity in Leech Giant Glial Cells

– Correspondence Analysis in Time-variant MicroscopicCorrespondence Analysis in Time-variant Microscopic3D Image Data3D Image Data

• Molecular DiagnosticsMolecular Diagnostics

– Genomics, Proteomics, Phylogenetic TreesGenomics, Proteomics, Phylogenetic Trees

Application AreasApplication AreasApplication AreasApplication Areas

• Projection SystemsProjection Systems– Concave Reality Projection SystemConcave Reality Projection System– RealityCenterRealityCenter– Interactive White BoardsInteractive White Boards– Active/Passive Stereo ProjectorsActive/Passive Stereo Projectors

• Spatial Tracking HardwareSpatial Tracking Hardware– Nest-of-Birds (electro-magnetic)Nest-of-Birds (electro-magnetic)– VICON Motion Tracking SystemVICON Motion Tracking System

(vision)(vision)

• Interaction HardwareInteraction Hardware– PinchGloves (data gloves)PinchGloves (data gloves)– Stylus (6dof pointing device)Stylus (6dof pointing device)

• Projection SystemsProjection Systems– Concave Reality Projection SystemConcave Reality Projection System– RealityCenterRealityCenter– Interactive White BoardsInteractive White Boards– Active/Passive Stereo ProjectorsActive/Passive Stereo Projectors

• Spatial Tracking HardwareSpatial Tracking Hardware– Nest-of-Birds (electro-magnetic)Nest-of-Birds (electro-magnetic)– VICON Motion Tracking SystemVICON Motion Tracking System

(vision)(vision)

• Interaction HardwareInteraction Hardware– PinchGloves (data gloves)PinchGloves (data gloves)– Stylus (6dof pointing device)Stylus (6dof pointing device)

InfrastructureInfrastructureInfrastructureInfrastructure

ConclusionsConclusionsConclusionsConclusions

• Engineering solutions can helpEngineering solutions can help

to make health care affordableto make health care affordable• Economic feasibilityEconomic feasibility• Data set size andData set size and

image quality increasesimage quality increases• Scalable solutionsScalable solutions

AcknowledgementsAcknowledgementsAcknowledgementsAcknowledgements• Center for Neuroscience,UC DavisCenter for Neuroscience,UC Davis

• Engineering Research Center, Mississippi State UniversityEngineering Research Center, Mississippi State University

• Center for Image Processing and Integrated ComputingCenter for Image Processing and Integrated Computing

(CIPIC), UC Davis(CIPIC), UC Davis

• The Scripps Research Institute (TSRI), La Jolla, CAThe Scripps Research Institute (TSRI), La Jolla, CA

• San Diego Supercomputer CenterSan Diego Supercomputer Center

• These projects are funded in part by the National Institute These projects are funded in part by the National Institute

of Mental Health (NIMH) through the Center for of Mental Health (NIMH) through the Center for

Neuroscience at UC Davis, and the National Partnership Neuroscience at UC Davis, and the National Partnership

for Advanced Computational Infrastructure (NPACI)for Advanced Computational Infrastructure (NPACI)

(award #10195430 00120410).(award #10195430 00120410).

• Center for Neuroscience,UC DavisCenter for Neuroscience,UC Davis

• Engineering Research Center, Mississippi State UniversityEngineering Research Center, Mississippi State University

• Center for Image Processing and Integrated ComputingCenter for Image Processing and Integrated Computing

(CIPIC), UC Davis(CIPIC), UC Davis

• The Scripps Research Institute (TSRI), La Jolla, CAThe Scripps Research Institute (TSRI), La Jolla, CA

• San Diego Supercomputer CenterSan Diego Supercomputer Center

• These projects are funded in part by the National Institute These projects are funded in part by the National Institute

of Mental Health (NIMH) through the Center for of Mental Health (NIMH) through the Center for

Neuroscience at UC Davis, and the National Partnership Neuroscience at UC Davis, and the National Partnership

for Advanced Computational Infrastructure (NPACI)for Advanced Computational Infrastructure (NPACI)

(award #10195430 00120410).(award #10195430 00120410).

That‘s all, folks!That‘s all, folks!That‘s all, folks!That‘s all, folks!

Joerg MeyerUniversity of California, IrvineBME/EECS Department644E Engineering TowerIrvine, CA 92697-2625

jmeyer@uci.eduhttp://www.eng.uci.edu/~jmeyer

Questions?

ucirvine department of biomedical engineering the state of the art joerg meyer jmeyer@uci.edu...

ucirvine biomedical

art joerg meyer

database slide

scaphoid bones slide

mri slide

portable system slide

b b c c d d slide

database solution

Documents

2015.11.30 platform clean ganges by joerg uehlin

controlling corruption problems, approaches and best...

joerg fritz demov4

150903 joerg hartmann - traeumen astronomen vom mann im mond

indicata e.n.g. automotive retail & distribution joerg...

© 2010 joerg woerner datamath calculator museum

lhc status tue morning 22-aug bernhard holzer, joerg...

joerg moellenkamp february 21th, 2010 -...

projects joerg stamm

© 2021 joerg woerner datamath calculator museum

integrated visualization and analysis of a pig's ... › pdf...

madonna harrington meyer - the maxwell … harrington meyer...

dr. joerg doerr - · dr. joerg doerr requirements ......

faith on the road by joerg rieger - excerpt

tetfusion: an algorithm for rapid tetrahedral mesh...

© 2020 joerg woerner datamath calculator museum

joerg tiede, assembly of state conferences/committee a...

evgenii v. kondratenko*, olga ovsitser, joerg radnik

michiel van noort nagaraju krishnappa joerg bischo · 2014....

joerg fritz demov3