21/08/12 1

A Web Interface for 3D Visualization and Interactive Segmentation of Medical

Images

H. Jacinto1,3, R. Kéchichian1, M. Desvignes2, R. Prost1, S. Valette1

1 : CREATIS2 : GIPSA-Lab3 : OneFit Medical SAS

21/08/12 2

Some of our previous works

1998 : 3D progressive transmission

- html- Netscape plugin (c++)- Cosmo Player

21/08/12 3

WebGL

2011 : Google BodyZygote Body

utf8 mesh encoder

→ Pure js is OK

65K limit

21/08/12 4

Context

Female Virtual HumanCT scan512*512*1734 = 455 MB

Output :RenderingProcessing (segmentation, detection, ...)

Input :Imaging data : MRI, CT ScannersLarge Volumetric Data

21/08/12 5

Medical visualization

MPRWhat physicians want

3D MeshesMedium Bandwidth

Remote Volume Renderinge.g. : ParaviewWebHigh bandwidthHigh server load

Volume RenderingHigh bandwidthHigh client load

MPR

Meshes

Volume Rendering

21/08/12 6

MPR

Server-side : Precompute 2D slices

● In-core● about 1:1 RAM → DISK trade● slicing delay (fixable)

Client-side : WebGL (three.js)● <img> → texture● shaders provide:

• Contrast/brightness• HDR• Lookup Tables

21/08/12 7

Under the hood

Server-side processing :

● Generic RPC framework● Bare filesystem as storage● RPC Caching

21/08/12 8

Data Formats

Voxel Data: DICOM → RAW/ZRAW →Slices

Surfaces : .vtk, .ply, .stl → OpenCTM (just workstm)

utf8, x3d are a possibility,But no web format is the end-goal

21/08/12 9

Interactive Segmentation

21/08/12 10

Graph cuts

[Boykov & Jolly 2001]

21/08/12 11

Graph cuts

Segmentation :Minimal cut

Multiple labels :α-expansion

21/08/12 12

Dealing with complexity

[Kéchichian et al. 2011]

Pixels → Clusters → Segmentation

21/08/12 13

Voronoi/Delaunay/Lloyd

Initial 2 iterations 50 iterations

E=∑i=1

N

∫V i

ρ(x ).d ( x , pi)2dx

21/08/12 14

Anatomical priors

Fat

PatelaCartilage

F

CartilageT

Tibia

Femur

Misc.Outside

Fibula

21/08/12 15

Results

21/08/12 16

Meshing

Marching Cubes :Too many triangles

21/08/12 17

Meshing

Approximated CentroidalVoronoi Diagrams[Valette & Chasserry 04]

21/08/12 18

Experiments on tritons/newts

MRI

Segmentation/meshing

- 20 individuals- 2 postures 'I' and 'S'

Collaboration with T. Wardziak (U. Lyon)

21/08/12 19

Arthroplasty

Orthopedics :The good angles

21/08/12 20

One Fit adapters

21/08/12 21

Some results

Reference Ours

21/08/12 22

Source code

ClientJs (8 kLOC) + qooxdoo + three.js

Server.php (1 kLOC) OR .js (1 kloc) + node.js

RPC (c++)- slicing/meshing :

github.com/valette/ACVDVTKOpenCTMGDCM

- segmentation : proprietary license

21/08/12 23

Thank you