june 11-13, 2002aqua question-answering system1 aqua: aquaint question answering system project...
TRANSCRIPT
June 11-13, 2002 AQUA Question-Answering System 1
AQUA: AQUAINT QuestionAQUA: AQUAINT QuestionAnswering SystemAnswering SystemProject Progress Report Project Progress Report
SAIC, San DiegoKSL, Stanford
June 11-13, 2002 AQUA Question-Answering System 3
Collaboration with NMSU TeamCollaboration with NMSU Team
This spring, NMSU has been added as an AQUAINT contractorSynergies between the NMSU Team and the SAIC Team have led to a collaborative effort for AQUAINT
June 11-13, 2002 AQUA Question-Answering System 4
SAIC-KSL-NMSU Collaborative SystemSAIC-KSL-NMSU Collaborative System
QUESTION
NL QueryInterlingua Query
KIF Q
uery
KIF Answer
Interlingua Answer
NL Answer
ANSWER
NMSU Query Processor
SAIC Interlingua KIF Translator
KSL Java Theorem Prover
SAIC KIF Interlingua Translator
NMSU NL Generator
June 11-13, 2002 AQUA Question-Answering System 5
Key Tasks—KSL Key Tasks—KSL
Providing knowledge tools for teamOntolingua Knowledge Server (OKS)Background ontologies and lexiconsJTP deductive answer determination system
Developing new methods forAnswer determination from large, complex KBs
• Using interoperating special-purpose reasoners • KB partitioning so that most reasoning occurs within partitions• Using Semantic Web representation languages (e.g., DAML+OIL)
Providing understandable explanations for deduced answers• Reasoning steps, data sourcessources, reasoning method, data
conversionsconversions, …• Use-specific and user-specific customization
Detecting and resolving conflicts in KBs• Proactive background testing for both likelylikely and provable
conflicts• Interactive tools for helping analyst correct or annotateannotate conflicts
June 11-13, 2002 AQUA Question-Answering System 6
Progress to Date—KSL Progress to Date—KSL
Developing JTP – An answer determining reasonerDeveloping a suite of special-purpose reasoners
E.g., for determining time-dependent answers
Reasoner for QA from ontological knowledge (in DAML+OIL)Produces and caches reasoning results during KB loadingWill be able to accept a series of queries, without waiting for answers
Usability improvements to JTPFor KB development and query-answering testing and debuggingSupport for rapid reloading and changing of developing KBs
E.g., Checkpointing and “untell”
Hierarchical presentation of reasoning explanationsDQL – Standard query language and QA protocol for DAML+OIL
Basic framework for client/server deductive query answering• Answers generated a batch at a time• Formal semantics
Applicable to other representation languagesBeing developed jointly with the DAML language design committeeWill be implemented for JTP
June 11-13, 2002 AQUA Question-Answering System 7
Key Tasks—SAIC Key Tasks—SAIC Ontolingua Translator
Automatic translation of information from NMSU team into KIF knowledge representations
• Includes dynamic semantic alignment of TMR Ontology and Ontolingua Ontology• Includes inferring relations that span multiple sentences• Includes flagging relations with document contexts (time, location)
Pre-reasoning to pre-answer known useful questions• Modular, interchangeable KB systems to determine specific types of relations
that are likely to be highly relevant and useful• Current system is event-based, but architecture valid for other types of
representations
OKS InterfaceTransfer, load, and store KIF representations into Ontolingua KBsMaintain DB of translated documents with info about contents of each
• Defines what documents are relevant to specific queries• Reduces burden of KB partitioning
– Each document has a separate KB file– Query TMR matched to contents of known documents to determine relevant KBs– Only KBs for relevant documents loaded into JTP to determine the answer
Source Credibility AssessmentDevelop historical records of source reliability for sourcesAssessment of estimated source credibility for incoming knowledgeDevelop methodologies for dynamic changes to credibility ratings
June 11-13, 2002 AQUA Question-Answering System 8
SAIC-KSL-NMSU Collaborative SystemSAIC-KSL-NMSU Collaborative System
QUESTION
NL QueryInterlingua Query
KIF Q
uery
KIF Answer
Interlingua Answer
NL Answer
ANSWER
NMSU Query Processor
SAIC Interlingua KIF Translator
KSL Java Theorem Prover
SAIC KIF Interlingua Translator
NMSU NL Generator
June 11-13, 2002 AQUA Question-Answering System 9
Converting Semantic Info to KnowledgeConverting Semantic Info to Knowledge
Ontologies (and ontological philosophies) differ between Ontolingua and TMR
SAIC dynamically aligns the two ontologies as part of the knowledge formation processMust be dynamic process since both TMR and Ontolingua ontologies are actively changing during system development
An automated mechanism for translating information in TMR into KIF relations is needed.
TMR breaks text into smallest piecesSAIC must re-unify the pieces to produce more meaningful knowledge representationsNext slide shows a typical example of this problem
June 11-13, 2002 AQUA Question-Answering System 10
Re-Unification of Knowledge Re-Unification of Knowledge
TMR includes separate references for this sentence fragment for:
united-states-soldiersoldier-human-adultunited-states-human-adult
We re-unify this into a single definition that captures that these are all the same and plural.
(defobject united-states-soldier (instance-of united-states-soldier person) (has-country united-states-soldier united-states)(member-of united-states-soldier us-army-special-
forces))(defobject group-of-united-states-soldier
(group group-of-united-states-soldier united-states-soldier)
(cardinality group-of-united-states-soldier 36))
SAMPLE TEXT: About 36 US Special Forces troops started a month of anti-terrorism training…
June 11-13, 2002 AQUA Question-Answering System 11
KIF Formulations from NarrativesKIF Formulations from Narratives
Often, useful text sources are narratives rather than unrelated compilations of factsStory comprehension must extend across sentences to the entire text of the narrativeSAIC’s experience developing knowledge representations from narratives in HPKB and RKF programs provides unique and powerful capability in this areaOur “event descriptor templates” provide relations needed to generate a series of event descriptors for a narrative
These allow us to answer highly complex questions about complicated, real-world situationsDeveloped and proven successful in HPKB ProgramNot trying to generate all possible relations from the text—only those relations that are in the event descriptors (i.e., known useful relations)
June 11-13, 2002 AQUA Question-Answering System 12
Why Event Templates?Why Event Templates?
Using event templates dramatically decreases the work load on JTP for each query
Distributes the analysis across multiple small reasoners specialized to answer specific types of questionsPre-query analysis anticipates common questions that may be asked about this document and pre-determines the answers automatically
• JTP’s set of multiple reasoners includes forward chaining, which may add other relations at load time rather than waiting for a specific query against the events
Result should be to dramatically improve answer response time for many queriesAlso provides extensibility of the system because each set of relations is handled by a separate, modular KB reasoner
• Support for other types of inputs than narratives may replace the mini-reasoners but doesn’t change the architecture
June 11-13, 2002 AQUA Question-Answering System 13
Progress to Date Progress to Date
Application of general-purpose rules that apply across a broad spectrum of instancesInitial processing of event basics in place
Identification of type of event, agent performing event, basic identification of object/agent acted on in event
• Subject-verb-object
Automatic definition of specific objects from textPlaces, people, groups
• Including cardinality of groups if available in original text
Location and time of eventDetermination (from raw text dateline, etc.) of event context location and event context timeAll events in this context are tagged relative to event-context-location and event-context-time
June 11-13, 2002 AQUA Question-Answering System 14
Automatic Knowledge Representation Status Automatic Knowledge Representation Status (cont.)(cont.)
More sophisticated event processingInterests relations
• General rules about actions supporting interests of agents performing them
• Citizenship relations
Action-object events• Dealing with verbs that are objects of actions in action-object-
is relations• Example: “pretending to do something” action-object-is
“doing something”• Representation is an event for “doing-something-event” and a
separate event for “pretending-the-doing-something-event”
Object re-unification• Recognition of previously referred to objects (from other
sentences or within longer sentences) as the same object• Pronoun dereferencing• Multiple phrasings of the same object
June 11-13, 2002 AQUA Question-Answering System 15
Status of Knowledge Representation SystemStatus of Knowledge Representation System
Preliminary demo of automatic translation from TMR to KIF is up and running
Our demonstration system is limited at the moment• Limited ontology• Limited relations• Limited lexical terms
The Demo System is improving quickly in its capabilities; in the meantime…