1 . 2 medical image processing, analysis & visualization in clinical research justin senseney...

1

http://mipav.cit.nih.gov

2

Medical Image Processing, Medical Image Processing, Analysis & Visualization Analysis & Visualization

in Clinical Researchin Clinical Research

Justin SenseneyJustin Senseney

SenseneyJ@mail.nih.govSenseneyJ@mail.nih.gov

dcb.cit.nih.gov/~senseneyjdcb.cit.nih.gov/~senseneyj

Biomedical Image Processing Research Services SectionBiomedical Image Processing Research Services Section

Center for Information TechnologyCenter for Information Technology

mipav.cit.nih.govmipav.cit.nih.gov

3

MIPAV TeamMIPAV TeamEmployeesEmployees

Ruida ChengRuida Cheng

William GandlerWilliam Gandler

Matthew McAuliffeMatthew McAuliffe

Evan McCreedyEvan McCreedy

Justin SenseneyJustin Senseney

FellowsFellows

Sara Shen (Maryland)Sara Shen (Maryland)

ContractorsContractors

Alexandra Bokinsky, Geometric Tools Inc. (Visualization)Alexandra Bokinsky, Geometric Tools Inc. (Visualization)

Olga Vovk, SRA International Inc. (Technical Writing)Olga Vovk, SRA International Inc. (Technical Writing)

AlumniAlumni

Paul Hemler (Hampden-Sydney), Agatha Monzon, Nishith Pandya (FITBIR), Paul Hemler (Hampden-Sydney), Agatha Monzon, Nishith Pandya (FITBIR),

Beth Tyriee (Kentucky), Hailong Wang (Heidelberg)Beth Tyriee (Kentucky), Hailong Wang (Heidelberg)

4

• PortabilityPortability– cross-platform or platform-independent executioncross-platform or platform-independent execution

• Data format independenceData format independence– access to images: DICOM, Analyze, TIFF, Raw, …access to images: DICOM, Analyze, TIFF, Raw, …

• ExtensibilityExtensibility – plugins and/or scriptsplugins and/or scripts

• ScalabilityScalability – foundation to support the growth to larger and more intricate data foundation to support the growth to larger and more intricate data

structuresstructures

• UsabilityUsability – coherent graphical user interface (GUI) coherent graphical user interface (GUI)

Requirements for an Image Quantification Requirements for an Image Quantification and Visualization Applicationand Visualization Application

5

Portability Portability Java PrimerJava Primer

Source Code Byte Code(class files)

Java Interpreterfor the PC

Machine code

PC

Java Interpreterfor a

Unix Workstation

Machine code

UNIXJava applications can be "written once and run anywhere", significantly reducing cross-platform development and maintenance.

6

Data Independence Data Independence

• DICOM file reader/writerDICOM file reader/writer• DICOM Query/Retrieve and “Catcher”DICOM Query/Retrieve and “Catcher”• Comprehensive file format support/conversionComprehensive file format support/conversion

– http://mipav.cit.nih.gov/fileformat.html

• MIPAV XML file format MIPAV XML file format

7

ExtensibilityExtensibility

Plugins and Scripts Plugins and Scripts • PluginsPlugins

– Functions written in Java using the MIPAV API. Functions written in Java using the MIPAV API.

• ScriptsScripts– Use MIPAV to record and save function(s) applied to Use MIPAV to record and save function(s) applied to

image dataset(s).image dataset(s).

– Apply the script to any number of image datasets Apply the script to any number of image datasets using the script wizard.using the script wizard.

8

• Model Image is an n-dimensional structure.Model Image is an n-dimensional structure.

• Algorithms can support up to 4D datasets.Algorithms can support up to 4D datasets.

• Viewers support 4D dataset with fusion.Viewers support 4D dataset with fusion.

ScalabilityScalability

9

UsabilityUsability

• GUI elementsGUI elements

• Scripting systemScripting system

• Command-line toolsCommand-line tools

10

11

File types(Raw, Analyze, DICOM 3.0, GE, Siemens, Bruker, Interfile,

Micro cat, MINC, MRC, FITS, Cheshire, AFNI, TIFF, JPEG, GIF, BMP, AVI, QuickTime, Biorad, Ziess LSM510, XML, and more)

Data (Image) types: n-dimensional structure(boolean, byte, unsigned byte, short,

unsigned short, int, long, float, double, Complex, ARGB)

Views – with data fusion 2D planar, “Lightbox”, Cine (movie), Multi-planar, 3D tri-planar, Surface render, (supports 3D texture mapped volume rendering Volume render

Algorithms Filtering Segmentation/classification Measurement/quantification Registration/fusion Utilities Plugins

VOIs

32KManual and automated contouring

Functional OverviewFunctional Overview

PACS DICOM 3.0:

Query/Retrieve, Catcher

GUI

Script ing

12

Opening ImagesOpening Images

13

Opening ImagesOpening Images

14

Image BrowserImage Browser

15

Opening ImagesOpening Images

16

Saving Image As (use suffix)Saving Image As (use suffix)

17

Code SnapshotCode Snapshot

int destExtents[] = new int[2];

destExtents[0] = image.getExtents()[0]; // X dim

destExtents[1] = image.getExtents()[1]; // Y dim

// Make a result image of Unsigned byte type

resultImage = new ModelImage(ModelStorageBase.UBYTE, destExtents, “Result Image”, null);

int length = destExtents[0] * destExtents[1];

for (int i = 0; i < length; i++){

destImage.set(i, i%256);

}

ViewJFrameImage imageFrame; ModelLUT LUTa = new ModelLUT(ModelLUT.COOLHOT, 256, dimExtentsLUT);

imageFrame = new ViewJFrameImage(resultImage, LUTa, new Dimension(610,200), userInterface);

18

Algorithms• Filters

• Calculation

• Registration

• Transformation

• Surface extraction

• Classification/Segmentation

19

Download and SetupDownload and Setup1. http://mipav.cit.nih.gov/download2. Fill in form3. Install (e.g. installMIPAV.exe)

** Nightly download - lastest changes but might have bugs.** Archived releases also available.

20

Memory AllocationMemory Allocation

General Rules• Do not exceed the computer’s physical RAM. For example if the computer has 1GB do not exceed approx 800MB.

• For 32-bit Windows systems do not exceed 1,400MB

21

Memory UsageMemory Usage

Press to recover memory

22

MIPAV Program OptionsMIPAV Program Options

23

Digital Image Communication in Medicine Digital Image Communication in Medicine (DICOM).(DICOM).

American College of Radiology (ACR) and the National ElectricalAmerican College of Radiology (ACR) and the National Electrical

Manufacturers Association (NEMA) formed a joint committee in 1983 to develop a Manufacturers Association (NEMA) formed a joint committee in 1983 to develop a

standard in Digital Image Communication in Medicine (DICOM). standard in Digital Image Communication in Medicine (DICOM).

1.1. Promote communication of digital image information, regardless of device manufacturerPromote communication of digital image information, regardless of device manufacturer

2.2. Facilitate the development and expansion of picture archiving and communication Facilitate the development and expansion of picture archiving and communication systems (PACS) that can also interface with other systems of hospital informationsystems (PACS) that can also interface with other systems of hospital information

3.3. Allow the creation of diagnostic information databases that can be interrogated by a wide Allow the creation of diagnostic information databases that can be interrogated by a wide variety of devices distributed geographically.variety of devices distributed geographically.

24

Patient

Database

DICOM Server

Images

Imaging Workstation (MIPAV) (PC, MAC, UNIX workstation)

Receiver

Query

Images

Image

Processing

&

Visualization

Imaging Device

DICOM ModelDICOM Model

PACS

Internet

25

DICOM DICOM communication communication

interfaceinterface

26

DICOMDICOM

27

DICOMDICOM

Access to image header information

28

DICOMDICOM AnonymizationAnonymization

29

DICOM File BrowserDICOM File Browser

30

XML Schema File FormatXML Schema File Format

31

XML Schema File FormatXML Schema File Format

32

XML Schema File FormatXML Schema File Format

33

Image AttributesImage Attributes

34

Volume of Interest (VOI)Volume of Interest (VOI)

35

VOIVOI

36

Lookup Table (LUT)Lookup Table (LUT)

37

Multi-planar and LightboxMulti-planar and Lightbox

38

Image FusionImage Fusion The loading of two images into the same frame

Controls blending between the two images

39

Structural MRI and Functional MRIStructural MRI and Functional MRI

40

Animation ToolAnimation Tool

41

Masks and SurfacesMasks and Surfaces

42

Scripting - RecordScripting - Record

43

Scripting - RunScripting - Run

44

HelpHelp

45

Bug ReportBug Report

46

MIPAVMIPAV

Visualization

Ubiquitous file reader

Quantification File writer

Processing Macros/Plugins

http://mipav.cit.nih.gov

SenseneyJ@mail.nih.gov

47