mustof - taking endoscopy to a higher dimension - a novel ... · visualisation of organs behind...

MUSTOF - taking endoscopy to a higher dimensionA novel 3-D hybrid endoscope for NOTES

January 30th, 2008

Dipl.-Ing. Kurt Höller,Dipl. Med.-Inf. Jochen PenneProf. Dr.-Ing. J. Hornegger

Institute of Pattern Recognition (Inf. 5)

Friedrich-Alexander-University Erlangen-Nuremberg

2 / 47

Content

1 Introduction/Motivation2 NOTES

Idea of NOTESChallenges with NOTES

3 MUSTOFTime-of-Flight (TOF)Idea of MUSTOFAlgorithmic frameworkLaser illumination

4 Summarize5 Outlook

Smart Biomedical IT, Nuremberg: K. Höller

3 / 47

Overview






4 / 47

Background of the project groupOur team: Multiple interests, one vision...

Organisational and personal infrastructure of the group:computer scientistselectrical engineersphysicistsphysicians

Industrial partners:endoscopycamerasoftware

Figure: In the operation roomSmart Biomedical IT, Nuremberg: K. Höller

5 / 47

First prototype of a 3-D endoscopebased on time-of-flight technology

first presented at


6 / 47

’Towards NOTES3D’Paketantrag

Participating institutes:LME, Erlangen (Prof. J. Hornegger)MITI group, Munich (Prof. H. Feussner)CAMP, Munich (Prof. N. Navab)MED1, Erlangen (Prof. E.G. Hahn)

Submitted during 3rd Russian-Bavarian Conference on BiomedicalEngineering


7 / 47

Overview






8 / 47

Overview






9 / 47

Time LineFrom open surgery to NOTES

Surgery can be done as:

open surgery→ for hundreds of years

minimally invasive / laparoscopicsurgery→ since the beginning of the 90s

and through natural orifices→ "no longer if but when" (W. O. Richards, D. W. Rattner 2005)

⇒ July 22/23, 2005 white paper and foundation of Consortiumfor Assesment and Research (NOSCAR) on NOTES:Natural Orifice Translumenal Endoscopic Surgery


10 / 47

NOTES PublicationsFast growing community

Figure: NOTES Publications in SE (SAGES), GIE (ASGE), Endoscopy (ESGE), DDW


11 / 47

Participating groups with NOTES

Figure: Interdisciplinarity of NOTES


12 / 47

Benefitsof Natural Orifice Translumenal Endosopic Surgery (NOTES)

Expected benefits of NOTES:

Less painFaster recoveryBetter cosmetic results avoiding skin incisionsLower risk for herniationNo risk for eventrationLower risk for wound infectionLower risk for adhesions


13 / 47

Peroral transgastric route


14 / 47

Flexible endoscope through wall of stomach

Figure: Resection of gastric stromal tumor (J.L. Ponsky 2006)Smart Biomedical IT, Nuremberg: K. Höller

15 / 47

Peranal transcolonic route


16 / 47

Transvaginal route


17 / 47

Peroral transesophageal route


18 / 47

Possible therapiesusing NOTES technique

Some actual discussed and tried therapies with NOTES:

appendectomycholecystectomysplenectomygastrojejunostomylymphadenectomynephrectomyliver biopsyhernia repair Figure: Splenectomy (Kantsevoy 2006)


19 / 47

Overview






20 / 47

Potential barriers to clinical practiceaccording to the NOTES white paper

Access to peritoneal cavity

⇒ item we could support

Gastric or intestinal closurePrevention of infectionDevelopment of suturing and anastomotic (nonsuturing) devicesMaintaining spatial orientation


Development of a multitasking platform


Management of intraperitoneal complications and hemorrhagePhysiologic untoward eventsTraining other providers


20 / 47

Potential barriers to clinical practiceaccording to the NOTES white paper

Access to peritoneal cavity ⇒ item we could supportGastric or intestinal closurePrevention of infectionDevelopment of suturing and anastomotic (nonsuturing) devicesMaintaining spatial orientation ⇒ item we could supportDevelopment of a multitasking platform ⇒ item we could supportManagement of intraperitoneal complications and hemorrhagePhysiologic untoward eventsTraining other providers


21 / 47

Access to peritoneal cavity: NOTES3D

Multi-Sensor-Time-Of-Flight (MUSTOF) technology enhances NOTESfor 3-D:

NOTES3D could be used to register online optic 3-D data withpreoperative MR or CT volumesSo combination of actual view of operation area with AugmentedReality (AR) is possibleThe optimal access point to peritoneal cavity can be found byvisualisation of organs behind gastric or colonic wall


22 / 47

Collision preventionPaketantrag ’Towards NOTES3D ’


23 / 47

Dynamic reconstructionPaketantrag ’Towards NOTES3D ’


24 / 47

Overview






25 / 47

Overview






26 / 47

State of the ArtTime-of-Flight (TOF) technology

Lateral resolution: 120×160 pixelDepth resolution: 3 mmWavelength: 870 nmPixel dimension: 40µm × 40µmModulation frequency: 20 Mhz (⇒ λ = 15m)Framerate: >12 fps

Figure: TOF-camera and example images


27 / 47

State of the ArtTime-of-Flight (TOF) technology

Lateral resolution: 176×144 pixelDepth resolution: 2,5 mmWavelength: 870 nmPixel dimension: 40µm × 40µmModulation frequency: 20 Mhz (⇒ λ = 15m)Framerate: >25 fps

Figure: TOF-camera and example images


28 / 47

Time-of-flight principleContinuous wave modulation


29 / 47

Modulation frequencyFrequency depending standard deviation


30 / 47

Compensation of temperature variation:Temperature depending error = bias (mesa)


31 / 47

Variation of integration time:Standard deviation and amplitude vs. integration time (pmd)

Comparison pmd/mesa:

Figure: integration time vs. amplitude and standard deviation(pmd)

Figure: integration time vs. amplitude and standard deviation(mesa)


32 / 47

Overview






33 / 47

Idea of MUSTOFParallel acquisition with TOF camera and CCD camera

Figure: Paketantrag ’Towards NOTES3D ’


34 / 47

Required MethodsCalibration and Registration of TOF camera and CCD camera



35 / 47

Required MethodsReconstruction of static or almost static 3-D scenes



36 / 47

Overview






37 / 47

Averaging of consecutive TOF framesResults

Test scenario: clinical object liver model close up

Figure: 3D surface reconstruction. Test scenario: Liver model. Left: original data. Right: result of averaging 10consecutive frames.


38 / 47

Outlier detection and removal by histogram analysisResults

Test scenario: clinical object liver model close up

Figure: 3D surface reconstruction. Test scenario: Liver model. Left: original data. Right: result of outlier removal.


39 / 47

Feature detectionResults

Test scenario: liver model close up

Figure: 3D surface reconstruction. Test scenario: liver model. Both images show the features detected in amplitude data(green) and 3D points (red).


40 / 47

Overview






41 / 47

Laser diodesAdvantage over a LED array

easier coupling into illumination chanelhigh powerfast modulationnarrow-band, ambient light suppression

Accuracy ∼ 1fmod

·√

Pmod + Pamb

P2mod


42 / 47

Experimental settingwith laser powered illumination


43 / 47

Overview






44 / 47

SummarizeApplications and Challenges

Supporting problems of NOTES will be THE application:

Access to peritoneal cavityRegistering online optic 3-D data with preoperative MR or CTvisualized by Augmented Reality

Maintaining spatial orientation, distance values or other 3-D datacollision prevention, motion compensation and automaticpositioning of surgery toolsreconstruction of static scenes (3-D mosaicing)

Research to make MUSTOF technology more precise:

modulation frequency and temperature depending offsetintegration time, modulation frequency and reflectivity dependingerrorshigh power laser illumination


45 / 47

Overview






46 / 47

Outlook

Next steps:

Spectra and power optimization of the MUSTOF system

Receiving data from animal laboratories (porcine model)

First results of Paketantrag ’Towards NOTES3D ’


47 / 47

The End

Thank you for your attention!

Any further questions?


mustof - taking endoscopy to a higher dimension - a novel ... · visualisation of organs behind...

Documents