PowerPoint-Prsentation

Frank HeckelFraunhofer MEVIS, Bremen, Germany | Innovation Center Computer Assisted Surgery, Leipzig, GermanySketch-Based Segmentation Editing for Oncological Therapy Monitoring

Fraunhofer MEVISMeeting of the Working Group VCBM, 3. September 2013, Vienna Fraunhofer MEVISNr. / 22Why do we need efficient segmentation editing tools?Segmentation is one of the essential tasks in medical image analysisMany sophisticated automatic segmentation algorithms exist which might fail in some cases (low contrast, noise, biological variability)

What to do?Manual segmentation? Takes too longDifferent algorithm? Might fail as wellLocally correct the error!

Fraunhofer MEVISNr. / 22The Segmentation Editing Process

Fraunhofer MEVISNr. / 22What makes segmentation editing a difficult problem?

Requirements:Intuitive interaction in 2D estimate the users intention in 3DLocal modificationsReal-time feedbackProvide a general tool (for different objects and modalities)Be independent of the preceding automatic algorithmThe user expects the tool to allow him or her to correct all errorsWith only a few steps!The segmentation problems are typically hard

Fraunhofer MEVISNr. / 22Editing ApproachesGuide initial algorithmDedicated editing tools Fraunhofer MEVISNr. / 22Sketch-Based Editing in 2D

User inputCorrection resultEdited regionPart containing the center of gravity Fraunhofer MEVISNr. / 22Sketch-Based Editing in 2DPossible Interactions

addremoveadd + removereplace Fraunhofer MEVISNr. / 22Sketch-Based Editing in 3DThe Correction Depth

Add/Remove:Replace: Fraunhofer MEVISNr. / 22Sketch-Based Editing in 3D

Image-Based Extrapolation

Fraunhofer MEVISNr. / 22Sketch-Based Editing in 3DImage-Based Extrapolation

s Fraunhofer MEVISNr. / 22Sketch-Based Editing in 3DImage-Independent Extrapolation

Fraunhofer MEVISNr. / 22Sketch-Based Editing in 3DPreviously performed corrections should be part of the new surfaceKeep all user-inputs and use them for reconstructionMajor issue: Contradictory inputsIgnore constraints from previous contours within a specific range:Image-Independent Extrapolation

1st input2nd input Fraunhofer MEVISNr. / 22Sketch-Based Editing in 3DImage-Independent Extrapolation

Fraunhofer MEVISNr. / 22Qualitative EvaluationEditing is a dynamic, user-driven processUser studies are the most important tool for evaluationQuality of editing tools is highly subjectiveSubjective quality suffers from bad intermediate results(Vague) criteria:AccuracyEfficiencyRepeatability

visually performed by the userintended result that the user only has in mind Fraunhofer MEVISNr. / 22Qualitative EvaluationRating Scheme for Accuracy and EfficiencyRatingMeaning++PerfectErrors could be corrected quickly with a few steps and according to the user's expectation.+GoodErrors could be corrected according to the user's expectation with slightly more effort, though.0AcceptableErrors could be corrected sufficiently at reasonable expense that would be acceptable in clinical routine. A few Intermediate results were unexpected, though.-BadCorrecting errors was complicated and took high effort. Several intermediate results were unexpected.--UnacceptableErrors could not be corrected or only with far to much effort. Many intermediate results were unexpected.sufficientinsufficient Fraunhofer MEVISNr. / 22Similar rating scheme for segmentation results

14Qualitative EvaluationResultsEditing rating score:

131 representative tumor segmentations in CT (lung nodules, liver metastases, lymph nodes)5 radiologists with different level of experience

Fraunhofer MEVISNr. / 22Quantitative Evaluation

Fraunhofer MEVISNr. / 22To have a comparable measure rather than a subjective rating,s_max = 5 in our context16Quantitative EvaluationResults

Fraunhofer MEVISNr. / 22110 = number of cases with reference segmentations (rating at least acceptable),Plot shows the median, the Q5- and Q95-quantiles and the outliers17Simulation-Based EvaluationProblem: High effort and bad reproducibility of user studiesIdea: Replace user by a simulationBenefits:Objective and reproducible validationObjective comparisonImproved regression testingBetter parameter tuning

Fraunhofer MEVISNr. / 22Simulation-Based EvaluationSimulationStep 1: Find most probably corrected 3D errorStep 2: Select slice and view where the error is most probably correctedStep 3: Generate user-input for sketchingStep 4: Apply editing algorithm

Fraunhofer MEVISNr. / 22Simulation-Based EvaluationResults

Fraunhofer MEVISNr. / 22Plot shows the median, the Q5- and Q95-quantiles and the outliers20Takeaway MessageThe segmentation problem is not solved yet

Segmentation editing is an indispensable step in the segmentation processParticularly for clinical routine, editing is a mandatory feature and not only nice to haveEfficient editing in 3D is challenging and not much work has been done in this field so far

Sketching has shown to be an intuitive interface in 2DThe presented methods provide efficient 3D editing tools for tumor segmentation Fraunhofer MEVISNr. / 22What's next?Editing for complex objects with irregular shapesSupport additional user inputsCombine image-based and image-independent approachesFurther improve the evaluation (measures, simulation)Find the most efficient human-computer interfaces for editing

Fraunhofer MEVISNr. / 22The last device is called phantom omni22

frank.heckel@mevis.fraunhofer.deThank you! Fraunhofer MEVIS Fraunhofer MEVISNr. / 22