Knowledge Base AccelerationTREC 2012

November 17, 2011

300BC 140AD 1828 1900 1950 1970 1984 1994 2001 now

Number of People Creating Representations of Knowledge

Library atAlexandria

Gutenberg Bible

Telegraph

Transistor

Machine Learning

Expert Systems

WWW

maps

November 17, 2011

Accelerate?

rate of assimilation << rate of new info

# editors << # active entities

(definition of a “large” KB)

November 17, 2011

Random Choicesnu

m p

ages

days

How many days must a news article wait before eventually being cited in Wikipedia?

Time lag in days between publication and eventual citation in Wikipedia of a sample of 50,000 web pages (mostly news) cited in Category:LivingPeople.

November 17, 2011

Even for entities mentioned frequently in the news, there is no correlation between mention and edit frequencies.

Human analysts follow their personal interests, hunches, hobbies, habits.

True for all large knowledge bases.

Even for entities mentioned frequently in the news, there is no correlation between mention and edit frequencies.

Human analysts follow their personal interests, hunches, hobbies, habits.

True for all large knowledge bases.

mea

n ed

it in

terv

al (d

ays)

mean mention interval (hours)

November 17, 2011

Methods in the Madnessm

ean

edit

inte

rval

(d

ays)

mean mention interval (hours)

November 17, 2011

First Year Task: Basic CCR“Cumulative Citation Recommendation”

Steps:1)Initialize with a single KB node:

1) Freebase & Wikipedia content2) WP edits from Aug-Nov 2011

2)Begin iterating over news stream3)For each article, output a “pertinence” confidence score between 0 and 1.

1) Aug-Sep: train on labels2) Oct-Mar: labels hidden

4)Your system generates labels and excerpts Oct-Mar

Content Stream•~500,000 English de-duplicated articles per day•Half news, half blogs & forums

Content Stream•~500,000 English de-duplicated articles per day•Half news, half blogs & forums

Your System

November 17, 2011

Gavin Rain•South African•Painter

Gavin Rain•South African•Painter

Venice Biennale(art show)Venice Biennale(art show)

Controversy about South African Pavilion at Venice Biennale

Controversy about South African Pavilion at Venice Biennale

will have an exhibit in

explicit mentions in newsexplicit mentions in news

explicit mentionsexplicit mentionsNo explicit co-occurrence

…inference?

No explicit co-occurrence

…inference?

1

2

3

Challenging Example

November 17, 2011

Annotations(guidelines under development)

November 17, 2011

Future Tasks

Detect changes to infobox slot values

Detect new links between entities

Resurrection of old articles (archive mining)

Identify emerging entities (not yet in KB)

… many more ideas …

November 17, 2011

TimelineDecember 2011Call for ParticipationTest data•Three nodes•Four months

March/April 2012Full data:•~50 nodes•Eight months

Monthly Skype Callsand discussion in Google Groups

Nov Dec Jan Feb Mar Apr Jun Jul Aug Sep Oct Nov

January 2012Tentative:•Eval results for baseline system•Data for more nodes

Summer meet up•At a convenient conference?

nowTREC2012

Submit your runs for eval

November 17, 2011

Optional Output Values(not judged in 2012)

• Novelty Group ID:– Output is a list of docIDs:

• Output an empty list means this doc has new information• Output one or more previous docIDs means that all of this document’s pertinent info was already

revealed in earlier docs

– Would help us plan future tasks about novelty

• Links to other nodes:– Output a list of other KB nodes that this content item associates to the target

node– Would help us plan future tasks about link detection

• Infobox slot name=value– Output a list of two-tuples of strings

• [(slot name, slot value),…]

– Would help us plan future tasks about detecting infobox changes

November 17, 2011