© 2009 endeca technologies, inc. all rights reserved. exploring semantic means daniel tunkelang...

29
© 2009 Endeca Technologies, Inc. All rights reserved. exploring semantic means Daniel Tunkelang Chief Scientist, Endeca

Upload: morris-lynch

Post on 17-Jan-2016

213 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: © 2009 Endeca Technologies, Inc. All rights reserved. exploring semantic means Daniel Tunkelang Chief Scientist, Endeca

© 2009 Endeca Technologies, Inc. All rights reserved.

exploring semantic means

Daniel TunkelangChief Scientist, Endeca

Page 2: © 2009 Endeca Technologies, Inc. All rights reserved. exploring semantic means Daniel Tunkelang Chief Scientist, Endeca

© 2009 Endeca Technologies, Inc. All rights reserved.2

about me

• 1988 – 1992

• 1993 – 1998

• 1999 -

Page 3: © 2009 Endeca Technologies, Inc. All rights reserved. exploring semantic means Daniel Tunkelang Chief Scientist, Endeca

© 2009 Endeca Technologies, Inc. All rights reserved.3

overview

a means, not an end

exploratory search (HCIR)

searching for meaning

Page 4: © 2009 Endeca Technologies, Inc. All rights reserved. exploring semantic means Daniel Tunkelang Chief Scientist, Endeca

© 2009 Endeca Technologies, Inc. All rights reserved.4

do you like our owl?

Page 5: © 2009 Endeca Technologies, Inc. All rights reserved. exploring semantic means Daniel Tunkelang Chief Scientist, Endeca

© 2009 Endeca Technologies, Inc. All rights reserved.5

a means, not an end

The Semantic Web will improve the connections between knowledge on the web and software.

Page 6: © 2009 Endeca Technologies, Inc. All rights reserved. exploring semantic means Daniel Tunkelang Chief Scientist, Endeca

© 2009 Endeca Technologies, Inc. All rights reserved.6

so much religion, so little time

Skeptical reactions

• Practical feasibility

• An unrealized idea

• Censorship and privacy

• Doubling output formats

• Need

vs.

Page 7: © 2009 Endeca Technologies, Inc. All rights reserved. exploring semantic means Daniel Tunkelang Chief Scientist, Endeca

© 2009 Endeca Technologies, Inc. All rights reserved.7

what semantic means to (people who pay) me

using linguistic and statistical approaches to derive meaning from unstructured text

using semantic web approaches to represent meaning in content and query structure

Page 8: © 2009 Endeca Technologies, Inc. All rights reserved. exploring semantic means Daniel Tunkelang Chief Scientist, Endeca

© 2009 Endeca Technologies, Inc. All rights reserved.8

exploratory search

what i really care about

what you should care about too

especially if you see more toinformation seeking than google

Page 9: © 2009 Endeca Technologies, Inc. All rights reserved. exploring semantic means Daniel Tunkelang Chief Scientist, Endeca

© 2009 Endeca Technologies, Inc. All rights reserved.9

what would google do?

information Need query select from results

rank using IR model

USER:

SYSTEM:tf-idf PageRank

Page 10: © 2009 Endeca Technologies, Inc. All rights reserved. exploring semantic means Daniel Tunkelang Chief Scientist, Endeca

© 2009 Endeca Technologies, Inc. All rights reserved.10

assumptions of relevance-centric approach

• self-awareness

• self-expression

• model knows best

• answer is a document

• one-shot query

Page 11: © 2009 Endeca Technologies, Inc. All rights reserved. exploring semantic means Daniel Tunkelang Chief Scientist, Endeca

© 2009 Endeca Technologies, Inc. All rights reserved.11

human-computer information retrieval

• don’t just guess the user’s intent– optimize communication

• increase user responsibility and control– require and reward human intellectual effort

“Toward Human-Computer Information Retrieval”

Gary Marchionini

Page 12: © 2009 Endeca Technologies, Inc. All rights reserved. exploring semantic means Daniel Tunkelang Chief Scientist, Endeca

© 2009 Endeca Technologies, Inc. All rights reserved.12

let’s make this concrete

• Colleague:

Hey Daniel! You should check out what this guy Steve Pollitt’s been researching. Sounds right up your alley.

• Daniel:

Sure thing, I’ll look into it.

Page 13: © 2009 Endeca Technologies, Inc. All rights reserved. exploring semantic means Daniel Tunkelang Chief Scientist, Endeca

© 2009 Endeca Technologies, Inc. All rights reserved.13

google him!

Page 14: © 2009 Endeca Technologies, Inc. All rights reserved. exploring semantic means Daniel Tunkelang Chief Scientist, Endeca

© 2009 Endeca Technologies, Inc. All rights reserved.14

google scholar him?

Page 15: © 2009 Endeca Technologies, Inc. All rights reserved. exploring semantic means Daniel Tunkelang Chief Scientist, Endeca

© 2009 Endeca Technologies, Inc. All rights reserved.15

hcir-inspired interface

Page 16: © 2009 Endeca Technologies, Inc. All rights reserved. exploring semantic means Daniel Tunkelang Chief Scientist, Endeca

© 2009 Endeca Technologies, Inc. All rights reserved.16

tags provide summarization and guidance

Page 17: © 2009 Endeca Technologies, Inc. All rights reserved. exploring semantic means Daniel Tunkelang Chief Scientist, Endeca

© 2009 Endeca Technologies, Inc. All rights reserved.17

my information need evolves as i learn

Page 18: © 2009 Endeca Technologies, Inc. All rights reserved. exploring semantic means Daniel Tunkelang Chief Scientist, Endeca

© 2009 Endeca Technologies, Inc. All rights reserved.18

how did we do that?

• wisdom of crowds

• corpus analysis

• scope the problem

• target exploratory search

Page 19: © 2009 Endeca Technologies, Inc. All rights reserved. exploring semantic means Daniel Tunkelang Chief Scientist, Endeca

© 2009 Endeca Technologies, Inc. All rights reserved.19

if you prefer sports to computer science

Page 20: © 2009 Endeca Technologies, Inc. All rights reserved. exploring semantic means Daniel Tunkelang Chief Scientist, Endeca

© 2009 Endeca Technologies, Inc. All rights reserved.20

a more current slide…

Page 21: © 2009 Endeca Technologies, Inc. All rights reserved. exploring semantic means Daniel Tunkelang Chief Scientist, Endeca

© 2009 Endeca Technologies, Inc. All rights reserved.21

hcir can get fancy

Page 22: © 2009 Endeca Technologies, Inc. All rights reserved. exploring semantic means Daniel Tunkelang Chief Scientist, Endeca

© 2009 Endeca Technologies, Inc. All rights reserved.22

and there’s query structure side of semantics

Page 23: © 2009 Endeca Technologies, Inc. All rights reserved. exploring semantic means Daniel Tunkelang Chief Scientist, Endeca

© 2009 Endeca Technologies, Inc. All rights reserved.23

a more current example

All reviewers Reviewers Under 18

Page 24: © 2009 Endeca Technologies, Inc. All rights reserved. exploring semantic means Daniel Tunkelang Chief Scientist, Endeca

© 2009 Endeca Technologies, Inc. All rights reserved.24

one last example

“…As an example, I may be interested in the tensions between government and newly nationalized banks over the bonus culture that taxpayers are now funding (a hot topic in the UK currently)…”

--a commenter at The Noisy Channel, in a discussion of exploratory search for news

Page 25: © 2009 Endeca Technologies, Inc. All rights reserved. exploring semantic means Daniel Tunkelang Chief Scientist, Endeca

© 2009 Endeca Technologies, Inc. All rights reserved.25

still in private beta…

Page 26: © 2009 Endeca Technologies, Inc. All rights reserved. exploring semantic means Daniel Tunkelang Chief Scientist, Endeca

© 2009 Endeca Technologies, Inc. All rights reserved.26

following the information scent

Page 27: © 2009 Endeca Technologies, Inc. All rights reserved. exploring semantic means Daniel Tunkelang Chief Scientist, Endeca

© 2009 Endeca Technologies, Inc. All rights reserved.27

to explore the news

Page 28: © 2009 Endeca Technologies, Inc. All rights reserved. exploring semantic means Daniel Tunkelang Chief Scientist, Endeca

© 2009 Endeca Technologies, Inc. All rights reserved.28

in closing

let’s not quibble over protocols

exploration is the killer app for semantic tech

it’s real, and it’s fabulous!

Page 29: © 2009 Endeca Technologies, Inc. All rights reserved. exploring semantic means Daniel Tunkelang Chief Scientist, Endeca

© 2009 Endeca Technologies, Inc. All rights reserved.29

thank you

communication 1.0email: [email protected]

communication 2.0blog: http://thenoisychannel.com

twitter: http://twitter.com/dtunkelang