Download - Enabling Natural Interaction
![Page 1: Enabling Natural Interaction](https://reader035.vdocument.in/reader035/viewer/2022062315/56815e3a550346895dcca061/html5/thumbnails/1.jpg)
Enabling Natural Interaction
Randall Davis, Howard ShrobeAaron Adler, Christine Alvarado, Mark Foltz,
Tracy Hammond, Mike Oltmans, Metin Sezgin,Olga Veselova
MIT Artificial Intelligence Lab
![Page 2: Enabling Natural Interaction](https://reader035.vdocument.in/reader035/viewer/2022062315/56815e3a550346895dcca061/html5/thumbnails/2.jpg)
Randall Davis
Consider This Device...
![Page 3: Enabling Natural Interaction](https://reader035.vdocument.in/reader035/viewer/2022062315/56815e3a550346895dcca061/html5/thumbnails/3.jpg)
Randall Davis
Natural Interaction?
• What if we had to deal with one another as we deal with our software?
![Page 4: Enabling Natural Interaction](https://reader035.vdocument.in/reader035/viewer/2022062315/56815e3a550346895dcca061/html5/thumbnails/4.jpg)
Randall Davis
Let’s Talk
![Page 5: Enabling Natural Interaction](https://reader035.vdocument.in/reader035/viewer/2022062315/56815e3a550346895dcca061/html5/thumbnails/5.jpg)
Randall Davis
Our Vision
![Page 6: Enabling Natural Interaction](https://reader035.vdocument.in/reader035/viewer/2022062315/56815e3a550346895dcca061/html5/thumbnails/6.jpg)
Randall Davis
Our Model
![Page 7: Enabling Natural Interaction](https://reader035.vdocument.in/reader035/viewer/2022062315/56815e3a550346895dcca061/html5/thumbnails/7.jpg)
Randall Davis
Rationale Capture
• Rationale capture doesn’t happen.– Because it’s too much trouble
• Rationale capture tools don’t work– Because they’re too much trouble
• The key: it needs to be less trouble than it’s worth.
• The key to less trouble: break down the barriers
![Page 8: Enabling Natural Interaction](https://reader035.vdocument.in/reader035/viewer/2022062315/56815e3a550346895dcca061/html5/thumbnails/8.jpg)
Randall Davis
Sketch Understanding
![Page 9: Enabling Natural Interaction](https://reader035.vdocument.in/reader035/viewer/2022062315/56815e3a550346895dcca061/html5/thumbnails/9.jpg)
Randall Davis
Why Sketching?
• Informal• Succinct• Natural• High in information content• Preserve topological and spatial
properties
![Page 10: Enabling Natural Interaction](https://reader035.vdocument.in/reader035/viewer/2022062315/56815e3a550346895dcca061/html5/thumbnails/10.jpg)
Randall Davis
Sketch Understanding
• Requires intelligence– at the signal level: early processing– at the interpretation level: handling ambiguity– at the engineering level: intended behavior
![Page 11: Enabling Natural Interaction](https://reader035.vdocument.in/reader035/viewer/2022062315/56815e3a550346895dcca061/html5/thumbnails/11.jpg)
Randall Davis
Other Research
• Additional domain: software diagrams• Multi-modal interaction• Describing and learning new icons• Generating efficient recognizers
![Page 12: Enabling Natural Interaction](https://reader035.vdocument.in/reader035/viewer/2022062315/56815e3a550346895dcca061/html5/thumbnails/12.jpg)
Randall Davis
Sketching Software
![Page 13: Enabling Natural Interaction](https://reader035.vdocument.in/reader035/viewer/2022062315/56815e3a550346895dcca061/html5/thumbnails/13.jpg)
Randall Davis
Sketching Software
![Page 14: Enabling Natural Interaction](https://reader035.vdocument.in/reader035/viewer/2022062315/56815e3a550346895dcca061/html5/thumbnails/14.jpg)
Randall Davis
Sketching Software
![Page 15: Enabling Natural Interaction](https://reader035.vdocument.in/reader035/viewer/2022062315/56815e3a550346895dcca061/html5/thumbnails/15.jpg)
Randall Davis
Sketching Software
![Page 16: Enabling Natural Interaction](https://reader035.vdocument.in/reader035/viewer/2022062315/56815e3a550346895dcca061/html5/thumbnails/16.jpg)
Randall Davis
Sketching Software//Source file: c:\\video\\Deck.javapublic class Deck extends Hand implements Dealable { public Card theCard; public Game theGame; /** * @roseuid 3C21348C0257 */ public Deck() { }}
//Source file: c:\\video\\Dealable.javapublic interface Dealable {}
//Source file: c:\\video\\Game.javapublic class Game { public Graphics theGraphics; public Deck theDeck; public Player thePlayer; /** * @roseuid 3C21348C02E3 */ public Game() { }}
![Page 17: Enabling Natural Interaction](https://reader035.vdocument.in/reader035/viewer/2022062315/56815e3a550346895dcca061/html5/thumbnails/17.jpg)
Randall Davis
Multi-Modal Interaction“Three identical, equally spaced pendulums…”
• Graphical vs verbal communication: What’s best said, what’s best sketched?
![Page 18: Enabling Natural Interaction](https://reader035.vdocument.in/reader035/viewer/2022062315/56815e3a550346895dcca061/html5/thumbnails/18.jpg)
Randall Davis
New Icons
• A shape description language• A drawing sequence description language
Define AndGateline L1 L2 L3arc Asemi-circle A1orientation(A1, 180)vertical L3parallel L1 L2same-horiz-position L1 L2connected A.p1 L3.p1connected A.p2 L3.p2 meets L1.p2 L3meets L2.p2 L3
![Page 19: Enabling Natural Interaction](https://reader035.vdocument.in/reader035/viewer/2022062315/56815e3a550346895dcca061/html5/thumbnails/19.jpg)
Randall Davis
Sketch Understanding Architecture
MechanicalEngineering
devicessimple compositionsprimitive objectsgeometry rectangles, curves
![Page 20: Enabling Natural Interaction](https://reader035.vdocument.in/reader035/viewer/2022062315/56815e3a550346895dcca061/html5/thumbnails/20.jpg)
Randall Davis
Sketch Understanding Architecture
MechanicalEngineering
devicessimple compositionsprimitive objects springs, jointsgeometry rectangles, curves
![Page 21: Enabling Natural Interaction](https://reader035.vdocument.in/reader035/viewer/2022062315/56815e3a550346895dcca061/html5/thumbnails/21.jpg)
Randall Davis
Sketch Understanding Architecture
MechanicalEngineering
devicessimple compositions ratchet, latchprimitive objects springs, jointsgeometry rectangles, curves
![Page 22: Enabling Natural Interaction](https://reader035.vdocument.in/reader035/viewer/2022062315/56815e3a550346895dcca061/html5/thumbnails/22.jpg)
Randall Davis
Sketch Understanding Architecture
MechanicalEngineering
devices mechanismssimple compositions ratchet, latchprimitive objects springs, jointsgeometry rectangles, curves
![Page 23: Enabling Natural Interaction](https://reader035.vdocument.in/reader035/viewer/2022062315/56815e3a550346895dcca061/html5/thumbnails/23.jpg)
Randall Davis
Sketch Understanding Architecture
Mechanical SoftwareEngineering Engineering
devices mechanismssimple compositions ratchet, latchprimitive objects springs, joints objects, methods
geometry rectangles, curves
![Page 24: Enabling Natural Interaction](https://reader035.vdocument.in/reader035/viewer/2022062315/56815e3a550346895dcca061/html5/thumbnails/24.jpg)
Randall Davis
Sketch Understanding Architecture
Mechanical SoftwareEngineering Engineering
devices mechanismssimple compositions ratchet, latch hierarchiesprimitive objects springs, joints objects, methods
geometry rectangles, curves
![Page 25: Enabling Natural Interaction](https://reader035.vdocument.in/reader035/viewer/2022062315/56815e3a550346895dcca061/html5/thumbnails/25.jpg)
Randall Davis
Sketch Understanding Architecture
Mechanical SoftwareEngineering Engineering
devices mechanisms modules
simple compositions ratchet, latch hierarchiesprimitive objects springs, joints objects, methods
geometry rectangles, curves
![Page 26: Enabling Natural Interaction](https://reader035.vdocument.in/reader035/viewer/2022062315/56815e3a550346895dcca061/html5/thumbnails/26.jpg)
Randall Davis
Design Rationale Capture
• Make it less trouble than it’s worth– Break down the barriers: eliminate
interfaces– Natural interaction is enabled by
intelligence• About sketching, gesturing• About the domain• About the task