cyclestack : inferring periodic behavior via temporal sequence visualization in ultrasound video
DESCRIPTION
CycleStack : Inferring Periodic Behavior via Temporal Sequence Visualization in Ultrasound Video. Teng-Yok Lee 1 , Abon Chaudhuri 1 Fatih Porikli 2 , Han-Wei Shen 1. 1 The Ohio State University. 2 Mitsubishi Electric Research Laboratories. Respiration-gated Therapy (1/2). - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: CycleStack : Inferring Periodic Behavior via Temporal Sequence Visualization in Ultrasound Video](https://reader035.vdocument.in/reader035/viewer/2022062500/5681500f550346895dbdf237/html5/thumbnails/1.jpg)
![Page 2: CycleStack : Inferring Periodic Behavior via Temporal Sequence Visualization in Ultrasound Video](https://reader035.vdocument.in/reader035/viewer/2022062500/5681500f550346895dbdf237/html5/thumbnails/2.jpg)
CycleStack: Inferring Periodic Behavior via Temporal Sequence Visualization in Ultrasound Video
Teng-Yok Lee1, Abon Chaudhuri1
Fatih Porikli2, Han-Wei Shen1
1The Ohio State
University
2Mitsubishi Electric Research
Laboratories
![Page 3: CycleStack : Inferring Periodic Behavior via Temporal Sequence Visualization in Ultrasound Video](https://reader035.vdocument.in/reader035/viewer/2022062500/5681500f550346895dbdf237/html5/thumbnails/3.jpg)
Respiration-gated Therapy (1/2) Treatment procedures need
accurate identification of the tumor’s location
Reducing the damage to the surrounding normal tissues
Increasing the efficiency of the treatment
The patient’s respiration can cause the periodic movement of the organs/tissues/tumors
![Page 4: CycleStack : Inferring Periodic Behavior via Temporal Sequence Visualization in Ultrasound Video](https://reader035.vdocument.in/reader035/viewer/2022062500/5681500f550346895dbdf237/html5/thumbnails/4.jpg)
Respiration-gated Therapy (2/2)
Utilizes the patient’s respiration signal to control the gate of the beam
Requires the therapist to verify the synchronization between the tissue’s motion and the respiration signal
Using ultrasound to sense the tissue’s motion
![Page 5: CycleStack : Inferring Periodic Behavior via Temporal Sequence Visualization in Ultrasound Video](https://reader035.vdocument.in/reader035/viewer/2022062500/5681500f550346895dbdf237/html5/thumbnails/5.jpg)
Requirements & Challenges
Detect the periodic cycles inside the (ultrasound) video
Burden in the short term visual memory
Visually verify the synchronization between the motion (of the tumor or tissues) and signal (respiration)
Different visual representation of the video and the signal
![Page 6: CycleStack : Inferring Periodic Behavior via Temporal Sequence Visualization in Ultrasound Video](https://reader035.vdocument.in/reader035/viewer/2022062500/5681500f550346895dbdf237/html5/thumbnails/6.jpg)
Post-therapy Analysis
For the purpose of quality assurance
Challenge
Watching video and signal can be time-consuming and tedious
Video visualization
Courtesy: Daniel and Chen, Video Visualization. In IEEE Vis ‘03
![Page 7: CycleStack : Inferring Periodic Behavior via Temporal Sequence Visualization in Ultrasound Video](https://reader035.vdocument.in/reader035/viewer/2022062500/5681500f550346895dbdf237/html5/thumbnails/7.jpg)
Our Approaches
Movement trace image Visualization of the motion in a video
Side-by-side visualization between the video and the signal
CycleStack Overview of the entire video and the signal
![Page 8: CycleStack : Inferring Periodic Behavior via Temporal Sequence Visualization in Ultrasound Video](https://reader035.vdocument.in/reader035/viewer/2022062500/5681500f550346895dbdf237/html5/thumbnails/8.jpg)
Movement Trace Image
Reveal the periodic cycles inside the video
Side-by-side comparison of the motion and the signal
time
Movement Trace Image
![Page 9: CycleStack : Inferring Periodic Behavior via Temporal Sequence Visualization in Ultrasound Video](https://reader035.vdocument.in/reader035/viewer/2022062500/5681500f550346895dbdf237/html5/thumbnails/9.jpg)
Movement Trace Image: Result
Synchronous time steps Synchronous time steps
Asynchronous case Opposite case
![Page 10: CycleStack : Inferring Periodic Behavior via Temporal Sequence Visualization in Ultrasound Video](https://reader035.vdocument.in/reader035/viewer/2022062500/5681500f550346895dbdf237/html5/thumbnails/10.jpg)
Movement Trace Image: Issue
Comparison in a sliding window
Too many cycles in the sliding window is not effective
![Page 11: CycleStack : Inferring Periodic Behavior via Temporal Sequence Visualization in Ultrasound Video](https://reader035.vdocument.in/reader035/viewer/2022062500/5681500f550346895dbdf237/html5/thumbnails/11.jpg)
CycleStack (1/2)
Preserves all time steps on the screen Overview of the video and signal
Organizes the movement trace image and signal into small multiples Efficient utilization of the screen space
![Page 12: CycleStack : Inferring Periodic Behavior via Temporal Sequence Visualization in Ultrasound Video](https://reader035.vdocument.in/reader035/viewer/2022062500/5681500f550346895dbdf237/html5/thumbnails/12.jpg)
CycleStack (2/2)
Input Respiratory Signal
2. Stack the cycles
Movement Trace Image
1. Segment the signal into periodic cycles
2. Stack the cycles
= CycleStack
![Page 13: CycleStack : Inferring Periodic Behavior via Temporal Sequence Visualization in Ultrasound Video](https://reader035.vdocument.in/reader035/viewer/2022062500/5681500f550346895dbdf237/html5/thumbnails/13.jpg)
Periodic Cycle Segmentation
1-threshold segmentation1-threshold segmentation of false cycles
![Page 14: CycleStack : Inferring Periodic Behavior via Temporal Sequence Visualization in Ultrasound Video](https://reader035.vdocument.in/reader035/viewer/2022062500/5681500f550346895dbdf237/html5/thumbnails/14.jpg)
CycleStack: Layout
Movement Trace Image
Segmented Cycles
CycleStack
Inhale Phases
Exhale Phases
Time
![Page 15: CycleStack : Inferring Periodic Behavior via Temporal Sequence Visualization in Ultrasound Video](https://reader035.vdocument.in/reader035/viewer/2022062500/5681500f550346895dbdf237/html5/thumbnails/15.jpg)
CycleStack: Asynchronous Cases
![Page 16: CycleStack : Inferring Periodic Behavior via Temporal Sequence Visualization in Ultrasound Video](https://reader035.vdocument.in/reader035/viewer/2022062500/5681500f550346895dbdf237/html5/thumbnails/16.jpg)
CycleStack: Opposite Cases
Synchronous cycles
Oppositecycles
![Page 17: CycleStack : Inferring Periodic Behavior via Temporal Sequence Visualization in Ultrasound Video](https://reader035.vdocument.in/reader035/viewer/2022062500/5681500f550346895dbdf237/html5/thumbnails/17.jpg)
Online CycleStack
Issue
Growing CycleStack can exceed the display window
Solution
CycleStack can be automatically rescaled to fit the display window
![Page 18: CycleStack : Inferring Periodic Behavior via Temporal Sequence Visualization in Ultrasound Video](https://reader035.vdocument.in/reader035/viewer/2022062500/5681500f550346895dbdf237/html5/thumbnails/18.jpg)
User Study: Setting (1/2)
Purpose Evaluate the response time and accuracy to detect non-
synchronous cases
Testing videos Asynchronous cases (1 synthesized & 1 ultrasound video)
Opposite cases (1 synthesized & 1 ultrasound video)
![Page 19: CycleStack : Inferring Periodic Behavior via Temporal Sequence Visualization in Ultrasound Video](https://reader035.vdocument.in/reader035/viewer/2022062500/5681500f550346895dbdf237/html5/thumbnails/19.jpg)
User Study: Setting (2/2)
18 participants in 2 groups One group watched videos and signals in separate
windows
The other group watched the CycleStack
Measurement Record the response time when the participants identified
non-synchronous cases
Compute the detection rate and average response time
![Page 20: CycleStack : Inferring Periodic Behavior via Temporal Sequence Visualization in Ultrasound Video](https://reader035.vdocument.in/reader035/viewer/2022062500/5681500f550346895dbdf237/html5/thumbnails/20.jpg)
User Study: Asynchronous Cases Observation about CycleStack
Higher detection rate in the ultrasound video
Slightly slower response time and larger standard deviation
Explanation
CycleStack keeps all cycles on the screen, allowing the viewer to detect the asynchronous cycle even after it has passed
Video Method Detection Rate Mean ±Std. Dev.
Synthesized Video/Signal 100% 1370.67±13.46
CycleStack 100% 1388.11±25.66
Ultrasound Video/Signal 44.4% 1887.00±32.72
CycleStack 88.9% 1966.38±141.83
![Page 21: CycleStack : Inferring Periodic Behavior via Temporal Sequence Visualization in Ultrasound Video](https://reader035.vdocument.in/reader035/viewer/2022062500/5681500f550346895dbdf237/html5/thumbnails/21.jpg)
User Study: Opposite Cases
Observation about CycleStack
Slightly slower response time but smaller standard deviation in the ultrasound video
Explanation
Opposite cases last longer than asynchronous cases
Some viewers of video and signal can detect it, even in late
Video Method Detection Rate Mean ±Std. Dev.
Synthesized Video/Signal 100% 1155.86±55.84
CycleStack 100% 1171.00±51.40
Ultrasound Video/Signal 88.9% 2488.88±206.06
CycleStack 100% 2557.00±102.36
![Page 22: CycleStack : Inferring Periodic Behavior via Temporal Sequence Visualization in Ultrasound Video](https://reader035.vdocument.in/reader035/viewer/2022062500/5681500f550346895dbdf237/html5/thumbnails/22.jpg)
Conclusion
Contribution An effective visualization that allows the viewer to compare
and to verify the synchronization between a video and a signal
Visual detection of non-synchronous cases during the online therapy and offline post-therapy analysis
Questions?