phase recognition during surgical procedures using embedded and body-worn sensors
DESCRIPTION
Presentation of paper on the PerCom 2011 conference in Seattle.TRANSCRIPT
![Page 1: Phase Recognition during Surgical Procedures using Embedded and Body-worn Sensors](https://reader034.vdocument.in/reader034/viewer/2022052623/5598b6bb1a28ab976c8b4788/html5/thumbnails/1.jpg)
Jakob Bardram, Afsaneh Doryab, Rune M.Jensen, Poul M. Lange, Kristian L. G.Nielsen, and Soeren T. Petersen
![Page 2: Phase Recognition during Surgical Procedures using Embedded and Body-worn Sensors](https://reader034.vdocument.in/reader034/viewer/2022052623/5598b6bb1a28ab976c8b4788/html5/thumbnails/2.jpg)
•Motivation•Field studies•Sensor platform•The experiment•The results•Discussion and Conclusion
Outline
![Page 3: Phase Recognition during Surgical Procedures using Embedded and Body-worn Sensors](https://reader034.vdocument.in/reader034/viewer/2022052623/5598b6bb1a28ab976c8b4788/html5/thumbnails/3.jpg)
Motivation
• Existing work has mainly focused on daily routines of an individual person in home or outdoor
![Page 4: Phase Recognition during Surgical Procedures using Embedded and Body-worn Sensors](https://reader034.vdocument.in/reader034/viewer/2022052623/5598b6bb1a28ab976c8b4788/html5/thumbnails/4.jpg)
MOTIVATIONHospital work is collaborative which involves constant coordination and communication
![Page 5: Phase Recognition during Surgical Procedures using Embedded and Body-worn Sensors](https://reader034.vdocument.in/reader034/viewer/2022052623/5598b6bb1a28ab976c8b4788/html5/thumbnails/5.jpg)
MOTIVATIONAutomatic recognition of surgical procedure can be used for:
•Coordination, communication and planning
•Patient safety
•Context-aware information management and retrieval
![Page 6: Phase Recognition during Surgical Procedures using Embedded and Body-worn Sensors](https://reader034.vdocument.in/reader034/viewer/2022052623/5598b6bb1a28ab976c8b4788/html5/thumbnails/6.jpg)
Motivation
How can we automatically recognize the phases of a surgery using sensors?
![Page 7: Phase Recognition during Surgical Procedures using Embedded and Body-worn Sensors](https://reader034.vdocument.in/reader034/viewer/2022052623/5598b6bb1a28ab976c8b4788/html5/thumbnails/7.jpg)
Motivation
How can we automatically recognize the phases of a surgery using sensors?
What should be sensed?
![Page 8: Phase Recognition during Surgical Procedures using Embedded and Body-worn Sensors](https://reader034.vdocument.in/reader034/viewer/2022052623/5598b6bb1a28ab976c8b4788/html5/thumbnails/8.jpg)
Motivation
How can we automatically recognize the phases of a surgery using sensors?
What should be sensed?Which sensors provide more significant input ?
![Page 9: Phase Recognition during Surgical Procedures using Embedded and Body-worn Sensors](https://reader034.vdocument.in/reader034/viewer/2022052623/5598b6bb1a28ab976c8b4788/html5/thumbnails/9.jpg)
What clinicians do in a surgery?
How do they move around?
Field study
![Page 10: Phase Recognition during Surgical Procedures using Embedded and Body-worn Sensors](https://reader034.vdocument.in/reader034/viewer/2022052623/5598b6bb1a28ab976c8b4788/html5/thumbnails/10.jpg)
4 important activity zones
![Page 11: Phase Recognition during Surgical Procedures using Embedded and Body-worn Sensors](https://reader034.vdocument.in/reader034/viewer/2022052623/5598b6bb1a28ab976c8b4788/html5/thumbnails/11.jpg)
Who is involved in a surgery?
Field study
![Page 12: Phase Recognition during Surgical Procedures using Embedded and Body-worn Sensors](https://reader034.vdocument.in/reader034/viewer/2022052623/5598b6bb1a28ab976c8b4788/html5/thumbnails/12.jpg)
Which tools and instruments they use?Where they use them?
Field study
![Page 13: Phase Recognition during Surgical Procedures using Embedded and Body-worn Sensors](https://reader034.vdocument.in/reader034/viewer/2022052623/5598b6bb1a28ab976c8b4788/html5/thumbnails/13.jpg)
Field study
• 97% of tasks involved at least one physical instrument and 78% involved several instruments
• Direct relation between physical instruments and physical tasks
![Page 14: Phase Recognition during Surgical Procedures using Embedded and Body-worn Sensors](https://reader034.vdocument.in/reader034/viewer/2022052623/5598b6bb1a28ab976c8b4788/html5/thumbnails/14.jpg)
Field study
• 97% of tasks involved at least one physical instrument and 78% involved several instruments
• Direct relation between physical instruments and physical tasks
![Page 15: Phase Recognition during Surgical Procedures using Embedded and Body-worn Sensors](https://reader034.vdocument.in/reader034/viewer/2022052623/5598b6bb1a28ab976c8b4788/html5/thumbnails/15.jpg)
Temporal and sequential procedure
![Page 16: Phase Recognition during Surgical Procedures using Embedded and Body-worn Sensors](https://reader034.vdocument.in/reader034/viewer/2022052623/5598b6bb1a28ab976c8b4788/html5/thumbnails/16.jpg)
•Based on our detailed study, these parameters seemed important to track:
![Page 17: Phase Recognition during Surgical Procedures using Embedded and Body-worn Sensors](https://reader034.vdocument.in/reader034/viewer/2022052623/5598b6bb1a28ab976c8b4788/html5/thumbnails/17.jpg)
•Based on our detailed study, these parameters seemed important to track:
•The location of clinicians and the patient
![Page 18: Phase Recognition during Surgical Procedures using Embedded and Body-worn Sensors](https://reader034.vdocument.in/reader034/viewer/2022052623/5598b6bb1a28ab976c8b4788/html5/thumbnails/18.jpg)
•Based on our detailed study, these parameters seemed important to track:
•The location of clinicians and the patient
•The location of objects on different tables
![Page 19: Phase Recognition during Surgical Procedures using Embedded and Body-worn Sensors](https://reader034.vdocument.in/reader034/viewer/2022052623/5598b6bb1a28ab976c8b4788/html5/thumbnails/19.jpg)
•Based on our detailed study, these parameters seemed important to track:
•The location of clinicians and the patient
•The location of objects on different tables
•The use of objects and instruments by clinicians
![Page 20: Phase Recognition during Surgical Procedures using Embedded and Body-worn Sensors](https://reader034.vdocument.in/reader034/viewer/2022052623/5598b6bb1a28ab976c8b4788/html5/thumbnails/20.jpg)
Sensor platform - Hardware
Ubisense server
Platform server
Arduino hub
ArduinoArm-wrist object tracking
UbisensePerson-tracking
AlienShort-range- object tracking
TCP Serial
TCP
TCP
USB
XBee
IcodeShort-range- object tracking
![Page 21: Phase Recognition during Surgical Procedures using Embedded and Body-worn Sensors](https://reader034.vdocument.in/reader034/viewer/2022052623/5598b6bb1a28ab976c8b4788/html5/thumbnails/21.jpg)
•Synchronize input from different sensors•store raw observations (one instance per second)
Sensor platform- Software
![Page 22: Phase Recognition during Surgical Procedures using Embedded and Body-worn Sensors](https://reader034.vdocument.in/reader034/viewer/2022052623/5598b6bb1a28ab976c8b4788/html5/thumbnails/22.jpg)
Sensor platform – Feature vector
![Page 23: Phase Recognition during Surgical Procedures using Embedded and Body-worn Sensors](https://reader034.vdocument.in/reader034/viewer/2022052623/5598b6bb1a28ab976c8b4788/html5/thumbnails/23.jpg)
Sensor Platform – Feature vector
![Page 24: Phase Recognition during Surgical Procedures using Embedded and Body-worn Sensors](https://reader034.vdocument.in/reader034/viewer/2022052623/5598b6bb1a28ab976c8b4788/html5/thumbnails/24.jpg)
The Experiment
![Page 25: Phase Recognition during Surgical Procedures using Embedded and Body-worn Sensors](https://reader034.vdocument.in/reader034/viewer/2022052623/5598b6bb1a28ab976c8b4788/html5/thumbnails/25.jpg)
The Experiment
![Page 26: Phase Recognition during Surgical Procedures using Embedded and Body-worn Sensors](https://reader034.vdocument.in/reader034/viewer/2022052623/5598b6bb1a28ab976c8b4788/html5/thumbnails/26.jpg)
The Experiment
![Page 27: Phase Recognition during Surgical Procedures using Embedded and Body-worn Sensors](https://reader034.vdocument.in/reader034/viewer/2022052623/5598b6bb1a28ab976c8b4788/html5/thumbnails/27.jpg)
The Experiment
![Page 28: Phase Recognition during Surgical Procedures using Embedded and Body-worn Sensors](https://reader034.vdocument.in/reader034/viewer/2022052623/5598b6bb1a28ab976c8b4788/html5/thumbnails/28.jpg)
![Page 29: Phase Recognition during Surgical Procedures using Embedded and Body-worn Sensors](https://reader034.vdocument.in/reader034/viewer/2022052623/5598b6bb1a28ab976c8b4788/html5/thumbnails/29.jpg)
![Page 30: Phase Recognition during Surgical Procedures using Embedded and Body-worn Sensors](https://reader034.vdocument.in/reader034/viewer/2022052623/5598b6bb1a28ab976c8b4788/html5/thumbnails/30.jpg)
Analysis
Test the feasibility of the sensor platform
![Page 31: Phase Recognition during Surgical Procedures using Embedded and Body-worn Sensors](https://reader034.vdocument.in/reader034/viewer/2022052623/5598b6bb1a28ab976c8b4788/html5/thumbnails/31.jpg)
Analysis
Test the feasibility of the sensor platformHow accurate phase recognition could be using sensed
data
![Page 32: Phase Recognition during Surgical Procedures using Embedded and Body-worn Sensors](https://reader034.vdocument.in/reader034/viewer/2022052623/5598b6bb1a28ab976c8b4788/html5/thumbnails/32.jpg)
Analysis
Test the feasibility of the sensor platformHow accurate phase recognition could be using sensed
dataIdentify the impact of each sensor in achieving high
accuracy
![Page 33: Phase Recognition during Surgical Procedures using Embedded and Body-worn Sensors](https://reader034.vdocument.in/reader034/viewer/2022052623/5598b6bb1a28ab976c8b4788/html5/thumbnails/33.jpg)
4 simulated operations – (4 datasets)Leave-one-out cross validationUsing decision trees for classification
Analysis - Phase Recognition
![Page 34: Phase Recognition during Surgical Procedures using Embedded and Body-worn Sensors](https://reader034.vdocument.in/reader034/viewer/2022052623/5598b6bb1a28ab976c8b4788/html5/thumbnails/34.jpg)
Analysis - Decision tree
![Page 35: Phase Recognition during Surgical Procedures using Embedded and Body-worn Sensors](https://reader034.vdocument.in/reader034/viewer/2022052623/5598b6bb1a28ab976c8b4788/html5/thumbnails/35.jpg)
Analysis – Initial Results
![Page 36: Phase Recognition during Surgical Procedures using Embedded and Body-worn Sensors](https://reader034.vdocument.in/reader034/viewer/2022052623/5598b6bb1a28ab976c8b4788/html5/thumbnails/36.jpg)
Analysis – Feature Set
Laryngoscope in anesthesia table zone
Anesthesia nurse in anesthesia machine zone
Tube in anesthesia table zone
… time
0 1 0 … t1
0 1 1 … t2
0 1 1 … t3
. . . . .
. . . . .
1 1 0 … tn
Features
Feature Instances at time t1-tn
![Page 37: Phase Recognition during Surgical Procedures using Embedded and Body-worn Sensors](https://reader034.vdocument.in/reader034/viewer/2022052623/5598b6bb1a28ab976c8b4788/html5/thumbnails/37.jpg)
Results – Phase RecognitionLaryngoscope in anesthesia table zone
Anesthesia nurse in anesthesia machine zone
Tube in anesthesia table zone
… time
0 1 0 … t1
0 1 1 … t2
0 1 1 … t3
. . . . .
. . . . .
1 1 0 … tn
Features
Feature Instances at time t1-tn
Laryngoscope in anesthesia table zone
Laryngoscope in anesthesia table zone- acc
Anesthesia nurse in anesthesia machine zone
Anesthesia nurse in anesthesia machine zone- acc
Tube in anesthesia table zone
Tube in anesthesia table zone- acc
… time
0 0 1 1 0 0 … t1
0 0 1 2 1 1 … t2
0 0 1 3 1 2 … t3
. . . . . .
. . . . . .
1 10 1 13 0 2 … tn
![Page 38: Phase Recognition during Surgical Procedures using Embedded and Body-worn Sensors](https://reader034.vdocument.in/reader034/viewer/2022052623/5598b6bb1a28ab976c8b4788/html5/thumbnails/38.jpg)
Feature Processing
Laryngoscope in anesthesia table
zone
Anesthesia nurse in anesthesia machine zone
Tube in anesthesia table zone
… time
0 1 0 … t1
0 1 1 … t2
0 1 1 … t3
. . . . .
. . . . .
1 1 0 … tn
![Page 39: Phase Recognition during Surgical Procedures using Embedded and Body-worn Sensors](https://reader034.vdocument.in/reader034/viewer/2022052623/5598b6bb1a28ab976c8b4788/html5/thumbnails/39.jpg)
Feature Processing
Laryngoscope in anesthesia table zone
laryngoscope in anesthesia table zone- accumulated
time
0 0 t1
1 1 t2
1 2 t3
0 2 t4
0 2 t5
1 3 t6
Laryngoscope in anesthesia table
zone
Anesthesia nurse in anesthesia machine zone
Tube in anesthesia table zone
… time
0 1 0 … t1
0 1 1 … t2
0 1 1 … t3
. . . . .
. . . . .
1 1 0 … tn
![Page 40: Phase Recognition during Surgical Procedures using Embedded and Body-worn Sensors](https://reader034.vdocument.in/reader034/viewer/2022052623/5598b6bb1a28ab976c8b4788/html5/thumbnails/40.jpg)
Results – Phase Recognition
With historical features
Without historical features
![Page 41: Phase Recognition during Surgical Procedures using Embedded and Body-worn Sensors](https://reader034.vdocument.in/reader034/viewer/2022052623/5598b6bb1a28ab976c8b4788/html5/thumbnails/41.jpg)
Sensor Significance
![Page 42: Phase Recognition during Surgical Procedures using Embedded and Body-worn Sensors](https://reader034.vdocument.in/reader034/viewer/2022052623/5598b6bb1a28ab976c8b4788/html5/thumbnails/42.jpg)
Sensor Significance
Ubisense only
![Page 43: Phase Recognition during Surgical Procedures using Embedded and Body-worn Sensors](https://reader034.vdocument.in/reader034/viewer/2022052623/5598b6bb1a28ab976c8b4788/html5/thumbnails/43.jpg)
Ubisense only
Tables+
wristband
Sensor Significance
![Page 44: Phase Recognition during Surgical Procedures using Embedded and Body-worn Sensors](https://reader034.vdocument.in/reader034/viewer/2022052623/5598b6bb1a28ab976c8b4788/html5/thumbnails/44.jpg)
Sensor Significance
Ubisense only
Tables+
Wristband
Wristbandsonly
![Page 45: Phase Recognition during Surgical Procedures using Embedded and Body-worn Sensors](https://reader034.vdocument.in/reader034/viewer/2022052623/5598b6bb1a28ab976c8b4788/html5/thumbnails/45.jpg)
Location important or not?Using palm based sensors in the OR?Using sensors or images?Classical machine learning enough?
Discussion
![Page 46: Phase Recognition during Surgical Procedures using Embedded and Body-worn Sensors](https://reader034.vdocument.in/reader034/viewer/2022052623/5598b6bb1a28ab976c8b4788/html5/thumbnails/46.jpg)
Conclusion
• Possible to achieve relatively high classification accuracy in phase recognition during surgical procedures using machine learning techniques
![Page 47: Phase Recognition during Surgical Procedures using Embedded and Body-worn Sensors](https://reader034.vdocument.in/reader034/viewer/2022052623/5598b6bb1a28ab976c8b4788/html5/thumbnails/47.jpg)
Conclusion
• Possible to achieve relatively high classification accuracy in phase recognition during surgical procedures using machine learning techniques
• The experiment hepled to analyze the weight and hence the importance of different sensors
![Page 48: Phase Recognition during Surgical Procedures using Embedded and Body-worn Sensors](https://reader034.vdocument.in/reader034/viewer/2022052623/5598b6bb1a28ab976c8b4788/html5/thumbnails/48.jpg)
Conclusion
• Possible to achieve relatively high classification accuracy in phase recognition during surgical procedures using machine learning techniques
• The experiment hepled to analyze the weight and hence the importance of different sensors
• Our study gives important input for further research in design of suitable sensors for the OR
![Page 49: Phase Recognition during Surgical Procedures using Embedded and Body-worn Sensors](https://reader034.vdocument.in/reader034/viewer/2022052623/5598b6bb1a28ab976c8b4788/html5/thumbnails/49.jpg)
Thank you