[siggraph2016 e-tech talk] laplacian vision: augmenting motion prediction via optical see-through...
TRANSCRIPT
- 1. Laplacian Vision: Yuta Itoh, Yuichi Hiroi, Jiu Otsuka, Maki Sugimoto, Jason Orlosky, Kiyoshi Kiyokawa, Gudrun Klinker Augmenting Motion Prediction via Optical See-Through Head-Mounted Displays and Projectors
- 2. Photo: Steve-65 Nave Physics (Hayes et al. 1978): our untrained perception of basic physical phenomena
- 3. In daily life Photo: AEQWF
- 4. Nave physics is, however, imperfect http://www.failepicfail.com/
- 5. In Science Fiction, a precog sees his short-term future Next, Nicholas Cage, TM & Paramount (2007)
- 6. or brain implants enable you to calculate it Jump 225 Trilogy, David Louis Edelman, 2006-2010 Wizards brain, Reiichi Saegusa, 2001-present
- 7. Laplace's demon: a causal determinism If the Demon knows the precise location and momentum of every atom in the universe, Essai philosophique sur les probabilits, Pierre-Simon Laplace, 1814 their past and future values for any given time are entailed; they can be calculated from the laws of classical mechanics.
- 8. Laplacian Vision: a vision augmentation system to assist motion prediction. Optical See-Through HMDShot directly through the system
- 9. Related work:Aftermath(AR + physics prediction) Leigh & Maes CHI15
- 10. Related work: projection mapping Koike & Yamaguchi, AH15 Projector Charette et al., ICCP12 Custom DLP projector Okumura et al., ICME12 Saccade Mirror
- 11. Related work: human vision enhancement Orlosky et al. TVCG15 Field of view Itoh & Klinker AH15 Eyesight X-ray vision Avey et al. VR09
- 12. Our goal is to assist our nave physics ability by augmenting our vision with an Optical See-Through HMD
- 13. We built a proof-of-concept system, and track a flying ball Tracking system User OST-HMD Virtual Image
- 14. We built a proof-of-concept system, and track a flying ball Tracking system
- 15. Projection mapping add-on by a projector
- 16. Projection mapping add-on by a projector
- 17. Our system estimates the future path of a flying ball in real time,
- 18. then visualizes an estimated path on the OST-HMD
- 19. Some views from the system when placed different positions
- 20. Some views from the system when placed different positions
- 21. Summary Assisting our nave physics skill Optical See-Through HMDs with a real-time AR rendering Simulating the physical behavior of the users environment
- 22. x Physics Human x OST-HMD Laplacian Vision: Yuta Itoh, Yuichi Hiroi, Jiu Otsuka, Maki Sugimoto, Jason Orlosky, Kiyoshi Kiyokawa, Gudrun Klinker Augmenting Motion Prediction via Optical See-Through Head-Mounted Displays and Projectors
- 23. Appendix
- 24. Example 1
- 25. Example 2
- 26. Example 3
- 27. User study: a fall-point prediction game With Laplacian vision Without (i.e. the nave vision) 29 subjects * 5 launches * with/without
- 28. 80% 90% Impact Video Length conditions 29 subjects * 5 launches * with/without * 3 lengths
- 29. 0 9045 View Angle conditions 29 subjects * 5 launches * with/without * 3 lengths * 3 angles
- 30. Result: 3x accuracy improvement 0 9045 Birds-eye view
- 31. Proposed Baseline 45 0 90 80% 90% 100% View Angle Video Length ** ** ** ** ** ** ** **
- 32. IR image (60Hz) Tracking (2x ARTTRACK2) 3D position of the ball (60Hz) Target ball System B Prediction & Rendering (Unity 3D) Real world System A OST-HMD (nVisor ST60) ~19 ms latency ~80 ms latency (User-view camera) Observer Displayed image (60Hz) Rendered image (60Hz) Latency diagram
- 33. Forward prediction 100 ms forward prediction No prediction Real ball carrying
- 34. HMD Display Calibration World Eye (a user-view camera)
- 35. Single Point Active Alignment Method (SPAAM) [Tuceryan 00] [Genc 02] Align a 3D point in the scene to 2D screen points A.R.T tracking system
- 36. Hardware