henry lieberman mit media lab ai in end-user software engineering henry lieberman, mit (presenting)...
TRANSCRIPT
Henry Lieberman • MIT Media Lab
AI in End-User Software Engineering
Henry Lieberman, MIT (presenting)
Raphael Hoffmann, Michael Toomim, U. Washington,
Seattle
Henry Lieberman • MIT Media Lab
Roles for AI in End-User SoftEng
Supply knowledge end-user doesn't have
Recognize user intent
Plan generation and plan recognition
Automate common patterns of use
Generalize & Specialize
Manage multiple representations
Error checking and sanity checking
Henry Lieberman • MIT Media Lab
U. Washington SoftBots & more
Planning-based software agents
Version-space Programming by Demonstration
Sequence prediction via Logic & Probability
Recommendation from software libraries
Henry Lieberman • MIT Media Lab
SmartEdit - Lau
Henry Lieberman • MIT Media Lab
Version Spaces
Henry Lieberman • MIT Media Lab
Linked Editing
Alternative to content abstraction facilities
Powerpoint Master Slides
Style Sheets
"Symbols" vs. instances in Flash
Web templates, components
User stays within WYSIWYG paradigm
Henry Lieberman • MIT Media Lab
Linked Editing
End-users "underuse" abstractions
Linked Editing
User edits concrete example
Generalizing over content rather than actions
Finds other instances of description
Performs "analogous" edits
Program code, Web pages, Vector graphics, Sheets
Henry Lieberman • MIT Media Lab
Linked Editing
Henry Lieberman • MIT Media Lab
Linked Editing
Henry Lieberman • MIT Media Lab
Assieme - Customization Recommendation
Goal: Recommending scripts for application extensions: Emacs, Firefox
Collect keystrokes, file names, commands
Provide Web service for recommendation
Collab filtering, learning classifier
Henry Lieberman • MIT Media Lab
Assieme - Customization Recommendation
Henry Lieberman • MIT Media Lab
The crazy dream: Programming in English
Programming languages are too difficult for human beings
People can express procedures in English (or other natural language)
Why not program in natural language?
That was the original idea behind Cobol
(whereas Fortran tried to make programming like math)
Henry Lieberman • MIT Media Lab
Childrens’ description of Pac-Man
Henry Lieberman • MIT Media Lab
But what can you do if you can only partially understand English?
Need to make the interaction fail-soft
Rely on dialogue to resolve ambiguities and errors
Maybe don’t eliminate code entirely
Emphasize correspondence between language and code
An "outliner" for code
Application: Programming MOOs
Henry Lieberman • MIT Media Lab
New resources for Common Sense Reasoning
Open Mind Common Sense knowledge base
ConceptNet Semantic Net
MontyLingua/LangUtils
Henry Lieberman • MIT Media Lab
Metafor Interface
Henry Lieberman • MIT Media Lab
Programmatic Semantics
English to “scaffolding” code
Roughly,
Turns nouns into classes and/or instances;
Verbs into functions
Adjectives into instance variables
Resolves anaphora
Untangles conditional and loops
Propagates context
Henry Lieberman • MIT Media Lab
Narrative stances
I want to make a bar with a customer. (1st person programmer)
There is a customer in the bar. (3rd person objective)
I am a customer sitting on a stool. (1st person program)
The bartender said, “Here is a customer” (3rd person subjective)
Henry Lieberman • MIT Media Lab
Refactoring
There is a bar. (atom)The bar contains two customers. (unimorphic list)It also contains a waiter. (unimorphic wrt. persons)It also contains some stools. (polymorphic list)The bar opens and closes. (class / agent)The bar is a kind of store. (inheritance class)Some bars close at 6pm. (subclass or instantiatable)
Henry Lieberman • MIT Media Lab
Shameless Plug