PowerPoint-Prsentation
Frank HeckelFraunhofer MEVIS, Bremen, Germany | Innovation Center Computer Assisted Surgery, Leipzig, GermanySketch-Based Segmentation Editing for Oncological Therapy Monitoring
Fraunhofer MEVISMeeting of the Working Group VCBM, 3. September 2013, Vienna Fraunhofer MEVISNr. / 22Why do we need efficient segmentation editing tools?Segmentation is one of the essential tasks in medical image analysisMany sophisticated automatic segmentation algorithms exist which might fail in some cases (low contrast, noise, biological variability)
What to do?Manual segmentation? Takes too longDifferent algorithm? Might fail as wellLocally correct the error!
Fraunhofer MEVISNr. / 22The Segmentation Editing Process
Fraunhofer MEVISNr. / 22What makes segmentation editing a difficult problem?
Requirements:Intuitive interaction in 2D estimate the users intention in 3DLocal modificationsReal-time feedbackProvide a general tool (for different objects and modalities)Be independent of the preceding automatic algorithmThe user expects the tool to allow him or her to correct all errorsWith only a few steps!The segmentation problems are typically hard
Fraunhofer MEVISNr. / 22Editing ApproachesGuide initial algorithmDedicated editing tools Fraunhofer MEVISNr. / 22Sketch-Based Editing in 2D
User inputCorrection resultEdited regionPart containing the center of gravity Fraunhofer MEVISNr. / 22Sketch-Based Editing in 2DPossible Interactions
addremoveadd + removereplace Fraunhofer MEVISNr. / 22Sketch-Based Editing in 3DThe Correction Depth
Add/Remove:Replace: Fraunhofer MEVISNr. / 22Sketch-Based Editing in 3D
Image-Based Extrapolation
Fraunhofer MEVISNr. / 22Sketch-Based Editing in 3DImage-Based Extrapolation
s Fraunhofer MEVISNr. / 22Sketch-Based Editing in 3DImage-Independent Extrapolation
Fraunhofer MEVISNr. / 22Sketch-Based Editing in 3DPreviously performed corrections should be part of the new surfaceKeep all user-inputs and use them for reconstructionMajor issue: Contradictory inputsIgnore constraints from previous contours within a specific range:Image-Independent Extrapolation
1st input2nd input Fraunhofer MEVISNr. / 22Sketch-Based Editing in 3DImage-Independent Extrapolation
Fraunhofer MEVISNr. / 22Qualitative EvaluationEditing is a dynamic, user-driven processUser studies are the most important tool for evaluationQuality of editing tools is highly subjectiveSubjective quality suffers from bad intermediate results(Vague) criteria:AccuracyEfficiencyRepeatability
visually performed by the userintended result that the user only has in mind Fraunhofer MEVISNr. / 22Qualitative EvaluationRating Scheme for Accuracy and EfficiencyRatingMeaning++PerfectErrors could be corrected quickly with a few steps and according to the user's expectation.+GoodErrors could be corrected according to the user's expectation with slightly more effort, though.0AcceptableErrors could be corrected sufficiently at reasonable expense that would be acceptable in clinical routine. A few Intermediate results were unexpected, though.-BadCorrecting errors was complicated and took high effort. Several intermediate results were unexpected.--UnacceptableErrors could not be corrected or only with far to much effort. Many intermediate results were unexpected.sufficientinsufficient Fraunhofer MEVISNr. / 22Similar rating scheme for segmentation results
14Qualitative EvaluationResultsEditing rating score:
131 representative tumor segmentations in CT (lung nodules, liver metastases, lymph nodes)5 radiologists with different level of experience
Fraunhofer MEVISNr. / 22Quantitative Evaluation
Fraunhofer MEVISNr. / 22To have a comparable measure rather than a subjective rating,s_max = 5 in our context16Quantitative EvaluationResults
Fraunhofer MEVISNr. / 22110 = number of cases with reference segmentations (rating at least acceptable),Plot shows the median, the Q5- and Q95-quantiles and the outliers17Simulation-Based EvaluationProblem: High effort and bad reproducibility of user studiesIdea: Replace user by a simulationBenefits:Objective and reproducible validationObjective comparisonImproved regression testingBetter parameter tuning
Fraunhofer MEVISNr. / 22Simulation-Based EvaluationSimulationStep 1: Find most probably corrected 3D errorStep 2: Select slice and view where the error is most probably correctedStep 3: Generate user-input for sketchingStep 4: Apply editing algorithm
Fraunhofer MEVISNr. / 22Simulation-Based EvaluationResults
Fraunhofer MEVISNr. / 22Plot shows the median, the Q5- and Q95-quantiles and the outliers20Takeaway MessageThe segmentation problem is not solved yet
Segmentation editing is an indispensable step in the segmentation processParticularly for clinical routine, editing is a mandatory feature and not only nice to haveEfficient editing in 3D is challenging and not much work has been done in this field so far
Sketching has shown to be an intuitive interface in 2DThe presented methods provide efficient 3D editing tools for tumor segmentation Fraunhofer MEVISNr. / 22What's next?Editing for complex objects with irregular shapesSupport additional user inputsCombine image-based and image-independent approachesFurther improve the evaluation (measures, simulation)Find the most efficient human-computer interfaces for editing
Fraunhofer MEVISNr. / 22The last device is called phantom omni22
[email protected] you! Fraunhofer MEVIS Fraunhofer MEVISNr. / 22