enabling natural interaction
DESCRIPTION
Enabling Natural Interaction. Randall Davis, Howard Shrobe Aaron Adler, Christine Alvarado, Mark Foltz, Tracy Hammond, Mike Oltmans, Metin Sezgin,Olga Veselova MIT Artificial Intelligence Lab. Consider This Device. Natural Interaction?. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Enabling Natural Interaction](https://reader035.vdocument.in/reader035/viewer/2022062315/56815e3a550346895dcca061/html5/thumbnails/1.jpg)
Enabling Natural Interaction
Randall Davis, Howard ShrobeAaron Adler, Christine Alvarado, Mark Foltz,
Tracy Hammond, Mike Oltmans, Metin Sezgin,Olga Veselova
MIT Artificial Intelligence Lab
![Page 2: Enabling Natural Interaction](https://reader035.vdocument.in/reader035/viewer/2022062315/56815e3a550346895dcca061/html5/thumbnails/2.jpg)
Randall Davis
Consider This Device...
![Page 3: Enabling Natural Interaction](https://reader035.vdocument.in/reader035/viewer/2022062315/56815e3a550346895dcca061/html5/thumbnails/3.jpg)
Randall Davis
Natural Interaction?
• What if we had to deal with one another as we deal with our software?
![Page 4: Enabling Natural Interaction](https://reader035.vdocument.in/reader035/viewer/2022062315/56815e3a550346895dcca061/html5/thumbnails/4.jpg)
Randall Davis
Let’s Talk
![Page 5: Enabling Natural Interaction](https://reader035.vdocument.in/reader035/viewer/2022062315/56815e3a550346895dcca061/html5/thumbnails/5.jpg)
Randall Davis
Our Vision
![Page 6: Enabling Natural Interaction](https://reader035.vdocument.in/reader035/viewer/2022062315/56815e3a550346895dcca061/html5/thumbnails/6.jpg)
Randall Davis
Our Model
![Page 7: Enabling Natural Interaction](https://reader035.vdocument.in/reader035/viewer/2022062315/56815e3a550346895dcca061/html5/thumbnails/7.jpg)
Randall Davis
Rationale Capture
• Rationale capture doesn’t happen.– Because it’s too much trouble
• Rationale capture tools don’t work– Because they’re too much trouble
• The key: it needs to be less trouble than it’s worth.
• The key to less trouble: break down the barriers
![Page 8: Enabling Natural Interaction](https://reader035.vdocument.in/reader035/viewer/2022062315/56815e3a550346895dcca061/html5/thumbnails/8.jpg)
Randall Davis
Sketch Understanding
![Page 9: Enabling Natural Interaction](https://reader035.vdocument.in/reader035/viewer/2022062315/56815e3a550346895dcca061/html5/thumbnails/9.jpg)
Randall Davis
Why Sketching?
• Informal• Succinct• Natural• High in information content• Preserve topological and spatial
properties
![Page 10: Enabling Natural Interaction](https://reader035.vdocument.in/reader035/viewer/2022062315/56815e3a550346895dcca061/html5/thumbnails/10.jpg)
Randall Davis
Sketch Understanding
• Requires intelligence– at the signal level: early processing– at the interpretation level: handling ambiguity– at the engineering level: intended behavior
![Page 11: Enabling Natural Interaction](https://reader035.vdocument.in/reader035/viewer/2022062315/56815e3a550346895dcca061/html5/thumbnails/11.jpg)
Randall Davis
Other Research
• Additional domain: software diagrams• Multi-modal interaction• Describing and learning new icons• Generating efficient recognizers
![Page 12: Enabling Natural Interaction](https://reader035.vdocument.in/reader035/viewer/2022062315/56815e3a550346895dcca061/html5/thumbnails/12.jpg)
Randall Davis
Sketching Software
![Page 13: Enabling Natural Interaction](https://reader035.vdocument.in/reader035/viewer/2022062315/56815e3a550346895dcca061/html5/thumbnails/13.jpg)
Randall Davis
Sketching Software
![Page 14: Enabling Natural Interaction](https://reader035.vdocument.in/reader035/viewer/2022062315/56815e3a550346895dcca061/html5/thumbnails/14.jpg)
Randall Davis
Sketching Software
![Page 15: Enabling Natural Interaction](https://reader035.vdocument.in/reader035/viewer/2022062315/56815e3a550346895dcca061/html5/thumbnails/15.jpg)
Randall Davis
Sketching Software
![Page 16: Enabling Natural Interaction](https://reader035.vdocument.in/reader035/viewer/2022062315/56815e3a550346895dcca061/html5/thumbnails/16.jpg)
Randall Davis
Sketching Software//Source file: c:\\video\\Deck.javapublic class Deck extends Hand implements Dealable { public Card theCard; public Game theGame; /** * @roseuid 3C21348C0257 */ public Deck() { }}
//Source file: c:\\video\\Dealable.javapublic interface Dealable {}
//Source file: c:\\video\\Game.javapublic class Game { public Graphics theGraphics; public Deck theDeck; public Player thePlayer; /** * @roseuid 3C21348C02E3 */ public Game() { }}
![Page 17: Enabling Natural Interaction](https://reader035.vdocument.in/reader035/viewer/2022062315/56815e3a550346895dcca061/html5/thumbnails/17.jpg)
Randall Davis
Multi-Modal Interaction“Three identical, equally spaced pendulums…”
• Graphical vs verbal communication: What’s best said, what’s best sketched?
![Page 18: Enabling Natural Interaction](https://reader035.vdocument.in/reader035/viewer/2022062315/56815e3a550346895dcca061/html5/thumbnails/18.jpg)
Randall Davis
New Icons
• A shape description language• A drawing sequence description language
Define AndGateline L1 L2 L3arc Asemi-circle A1orientation(A1, 180)vertical L3parallel L1 L2same-horiz-position L1 L2connected A.p1 L3.p1connected A.p2 L3.p2 meets L1.p2 L3meets L2.p2 L3
![Page 19: Enabling Natural Interaction](https://reader035.vdocument.in/reader035/viewer/2022062315/56815e3a550346895dcca061/html5/thumbnails/19.jpg)
Randall Davis
Sketch Understanding Architecture
MechanicalEngineering
devicessimple compositionsprimitive objectsgeometry rectangles, curves
![Page 20: Enabling Natural Interaction](https://reader035.vdocument.in/reader035/viewer/2022062315/56815e3a550346895dcca061/html5/thumbnails/20.jpg)
Randall Davis
Sketch Understanding Architecture
MechanicalEngineering
devicessimple compositionsprimitive objects springs, jointsgeometry rectangles, curves
![Page 21: Enabling Natural Interaction](https://reader035.vdocument.in/reader035/viewer/2022062315/56815e3a550346895dcca061/html5/thumbnails/21.jpg)
Randall Davis
Sketch Understanding Architecture
MechanicalEngineering
devicessimple compositions ratchet, latchprimitive objects springs, jointsgeometry rectangles, curves
![Page 22: Enabling Natural Interaction](https://reader035.vdocument.in/reader035/viewer/2022062315/56815e3a550346895dcca061/html5/thumbnails/22.jpg)
Randall Davis
Sketch Understanding Architecture
MechanicalEngineering
devices mechanismssimple compositions ratchet, latchprimitive objects springs, jointsgeometry rectangles, curves
![Page 23: Enabling Natural Interaction](https://reader035.vdocument.in/reader035/viewer/2022062315/56815e3a550346895dcca061/html5/thumbnails/23.jpg)
Randall Davis
Sketch Understanding Architecture
Mechanical SoftwareEngineering Engineering
devices mechanismssimple compositions ratchet, latchprimitive objects springs, joints objects, methods
geometry rectangles, curves
![Page 24: Enabling Natural Interaction](https://reader035.vdocument.in/reader035/viewer/2022062315/56815e3a550346895dcca061/html5/thumbnails/24.jpg)
Randall Davis
Sketch Understanding Architecture
Mechanical SoftwareEngineering Engineering
devices mechanismssimple compositions ratchet, latch hierarchiesprimitive objects springs, joints objects, methods
geometry rectangles, curves
![Page 25: Enabling Natural Interaction](https://reader035.vdocument.in/reader035/viewer/2022062315/56815e3a550346895dcca061/html5/thumbnails/25.jpg)
Randall Davis
Sketch Understanding Architecture
Mechanical SoftwareEngineering Engineering
devices mechanisms modules
simple compositions ratchet, latch hierarchiesprimitive objects springs, joints objects, methods
geometry rectangles, curves
![Page 26: Enabling Natural Interaction](https://reader035.vdocument.in/reader035/viewer/2022062315/56815e3a550346895dcca061/html5/thumbnails/26.jpg)
Randall Davis
Design Rationale Capture
• Make it less trouble than it’s worth– Break down the barriers: eliminate
interfaces– Natural interaction is enabled by
intelligence• About sketching, gesturing• About the domain• About the task