1 context machine perry mcdowellmichael capps mike zydaandrzej kapolka
TRANSCRIPT
1
Context Machine
Perry McDowell Michael Capps
Mike Zyda Andrzej Kapolka
2
Outline
The Context Machine
Literary Inspirations
Payoff and Impact
Goals and Objectives
Technical Approach
Accomplishments
Demo
Future Work
3
Literary Inspirations
Diamond Age, Neal Stephenson
Futuristic novel of a young girl who discovers an interactive book which plays scenes demonstrating proper behavior of a young lady
She is able to drastically change her life by adhering to lessons learned in the book
4
Literary Inspirations
Starship Troopers, Robert A. Heinlein Futuristic novel of a young man who joins
the Mobile Infantry and fights in a intergalatic war
The soldiers wear an automated suit which greatly enhances their lethality and survivability
Goal is to give soldiers the same improvement mentally that the suits gave physically
5
The Context Machine
We are building a Context Machine that determines the “context” of a situation for the user and provides advice/guidance.
We anticipate that this augmentation will become essential to future soldiers.
6
How the Context Machine Helps Users
The context machine is envisioned to improve flow of information to users in several ways: Informing users of things they can not observe Informing and filtering for users things they don’t have the
time to observe Explaining situations and devices users can’t understand Providing an perceptive interface for the user
7
Where Does the Context Machine Fit in AugCog?
LocationLocation
ActionsActions
PeoplePeople
ObjectsObjects
Outer World Outer World AbstractionAbstraction
Determine Determine Context Context and COAand COA
Output to Output to UserUser
User’s User’s CommandsCommands
Goal 1Goal 1
Goal 4Goal 4 Goal nGoal n
Goal 2Goal 2
Goal 3Goal 3
PreferencesPreferences
Visual Visual DisplayDisplay
Aural Aural DisplayDisplay
8
Payoff/Impact
A mechanism for accepting symbolic information from a sensory stream and inserting it into a robust dynamic architecture for determining and representing context
Provides expert advice in limited domains
Capable of learning new and novel situations
Monitors user’s goals and assists where appropriate
Adaptable to individual users
9
How to Build the Middle Without the Ends
We used a game engine to create a world where we could test the Context Machine’s ability to determine context The engine simulates the system that extracts
data from the real world and puts it into the symbolic information
Game Engine Game Engine WorldWorld
Game Engine Game Engine DisplayDisplay
Context Context MachineMachine
10
And Now, a Word From My Sponsor…
MOVES Institute has a new War Games Lab Uses game technology to rapidly build high fidelity simulations
20 artists and programmers from the game industry Plus plenty of cheap graduate student labor
Work closely with numerous game companies Unlimited license for Epic Games’ UNREAL engine Currently building products or demonstrations for US Army,
SOCOM, DTRA, DOE, ONR, NRO, and others Vague website at http://www.wargameslab.com/
12
Technical Approach
We are pursuing a concurrent and intertwined two-tracked approach: Track 1 involves developing hypothesis of
context Literature search of current research into context Develop a taxonomy of context elements Define “necessary and sufficient” for creating context-
aware applications Track 2 involves creating context-aware
applications to test our hypotheses These applications begin simply in specific domains
and increase in complexity and generality
13
What We Have Accomplished
Wrote a technical report which consists of an in-depth literature review of the research previously done on context-aware systems. Summarizes the major efforts in context-
aware computing to date Presents both the definitions of context and
the attributes used to determine context Determines a taxonomy to be used in future
investigations into context
14
What We Have Accomplished
Created a simulation demonstrating assistance furnished by a Context Machine Acts as a base for our later investigations Demonstrated rudimentary context sensing
capability Demonstrated ways the Context Machine can
increase cognitive throughput Serves as a test platform to test techniques to
improve the user’s cognition
15
Specific Accomplishments
Created a pathing system which can determine the best path for a user to follow
Created an imperfect mental model to reflect the facts known by the Context Machine
Use that information to determine context and advise user of appropriate course of action
16
Demo
17
Demo
Goal of the simulation is to transit the treeline on the left of the camp, go to the roof of the building (“A”) and observe a convoy passing down the road to the right.
A
Ingress Convoy Location
18
Directional Guidance
Arrows provide directional guidanceIntend to add other information to arrows
19
Improving Recognition
Augmented reality cursor points out important information that might otherwise not be noticed
20
Displays Information Regarding Mental Model
Bar indicates degree of certainty as to the identification of an object/person in the Context Machine’s mental model
21
Remembers Information
After identifying objects, the Context Machine remembers them so user can concentrate on other things
22
Finds Paths Based Upon Current Info
Pathing can be based on a wide variety of criteria
24
Contact Information
Perry McDowellThe MOVES InstituteNaval Postgraduate School833 Dyer Road, Code CS/mpMonterey, CA 93943-5118(831) [email protected]://www.movesinstitute.org/~augcog/