natural language interfaces to ontologies danica damljanović danica@dcs.shef.ac.uk

Natural Language Interfaces to Ontologies

Danica Damljanović

danica@dcs.shef.ac.uk

University of Sheffield NLP

Outline

• NLIs to ontologies and their usability

• QuestIO – Question-based Interface to Ontologies

• Towards better usability using FREyA• Demo and evaluation

• Conclusion

210/12/09 Danica Damljanović

University of Sheffield NLP

Motivation

Large datasets such as Linked Open Data available

SPARQL/SeRQL: complex syntax: not easy to learn writing queries is error-prone task requires understanding of Semantic Web

technologies

University of Sheffield NLP

Objective

Allow domain experts to query knowledge in RDF/OWL format in a user-friendly manner

University of Sheffield NLP

Danica Damljanović

Natural Language user-friendly?

(Kaufmann and Bernstein, 2007)• Natural Language Interfaces preferred to keywords, menu-

guided, and graphical interfaces

(Linckels, 2007): • keywords preferred to NL interfaces

University of Sheffield NLP

6Danica Damljanović

NLIs to ontologies

University of Sheffield NLP

Error rate caused by…

• Users not familiar with the covered knowledge

• Knowledge is not available, but the system is not making that clear to the user i.e. feedback messages not helpful

• Users have assumptions/misconceptions about the system capabilities and supported language

University of Sheffield NLP

8Danica Damljanović

Usable NLIs to KBs: challenges

• Robustness

• Portability

• What to show?

• Understanding information need

• Habitability

University of Sheffield NLP

Challenging habitability problem

University of Sheffield NLP

Question-based Interface to Ontologies

• Robust• Zero customisation!• Easy to use: no training for

the user.• Deal with incorrectly

formulated queries• Accept queries of any

length and form.• Automatically ranks results

and shows the highest rank to the user

University of Sheffield NLP

NL --> SeRQL query

Filtering concepts

Ranking concepts

Query Creator

Query Execution

University of Sheffield NLP

An Example

1.15

1.19

compare

University of Sheffield NLP

FREyA (Feedback, Refinement, Extended Vocabulary Aggregator)

• assist the user formulate the query and express his need more precisely

• Implement user-system interaction and learn from it

University of Sheffield NLP

University of Sheffield NLP

Example 1

geo:City

geo:State new york

POC

POC

population

geo:cityPopulation

University of Sheffield NLP

New York is a city

University of Sheffield NLP

New York is a state

University of Sheffield NLP

POC

POC

POC

state

area geo:stateArea geo:State

geo:isLowestPointOf

lowest point

Example 2

University of Sheffield NLP

Demo

• http://gate.ac.uk/freya

University of Sheffield NLP

In lab evaluation• Mooney dataset: geography of the United States

• Relatively small ontology

• 250 questions

• require a high level of understanding of semantic meaning

• Recall and precision:

• 76% when we identify only the answer type correctly (first POC in the question)

• Expected: ~91% when we include all POCs from the question and add more refined suggestions (such as max or min for datatype properties of type number)

University of Sheffield NLP

Next steps

• Improvement of the learning mechanism

• User-based evaluation

• Trying it with some other datasets

University of Sheffield NLP

Thank you!

• Questions?