advance information retrieval topics

Advance Information Retrieval Topics

Hassan Bashiri

Information Filtering

Agenda

• Information filtering

• Automatic profile learning

• Social filtering

• Training Strategies

Information Access Problems

Collection

Info

rmat

ion

Nee

d

Stable

Stable

DifferentEach Time

DataMining

Retrieval

Filtering

DifferentEach Time

Indexing and Complexity

Agenda

• Inverted indexes

• Computational complexity

An Example

quick

brown

fox

over

lazy

dog

back

now

time

all

good

men

come

jump

aid

their

party

00110000010010110

01001001001100001

Term Doc

1

Doc

2

00110110110010100

11001001001000001

Doc

3D

oc 4

00010110010010010

01001001000101001

Doc

5D

oc 6

00110010010010010

10001001001111000

Doc

7D

oc 8

A

B

C

FD

GJLMNOPQ

T

AIALBABR

THTI

4 ,82 ,4 ,61 ,3 ,7

1 ,3 ,5 ,72 ,4 ,6 ,8

3 ,53 ,5 ,7

2 ,4 ,6 ,83

1 ,3 ,5 ,72 ,4 ,82 ,6 ,8

1 ,3 ,5 ,7 ,86 ,81 ,31 ,5 ,72 ,4 ,6

PostingsInverted File

The Finished Product

quick

brown

fox

over

lazy

dog

back

now

time

all

good

men

come

jump

aid

their

party

Term

A

B

C

FD

GJLMNOPQ

T

AIALBABR

THTI

4 ,82 ,4 ,61 ,3 ,7

1 ,3 ,5 ,72 ,4 ,6 ,8

3 ,53 ,5 ,7

2 ,4 ,6 ,83

1 ,3 ,5 ,72 ,4 ,82 ,6 ,8

1 ,3 ,5 ,7 ,86 ,81 ,31 ,5 ,72 ,4 ,6

PostingsInverted File

Cross-Language Information Retrieval

Agenda

• Cross-language IR– Controlled vocabulary

• Automatic indexing

– Free text– Evaluation– User interface design

11

What is CLIR?

Users enter their query in one language and the search engine retrieves relevant documents in other languages.

Retrieval SystemEnglish Query French Documents

11

Term-aligned Sentence-aligned Document-aligned Unaligned

Parallel Comparable

Knowledge-based Corpus-based

Controlled Vocabulary Free Text

Cross-Language Text Retrieval

Query Translation Document Translation

Text Translation Vector Translation

Ontology-based Dictionary-based

Thesaurus-based

Retrieval System Interfaces

Agenda

• Query interface

• Selection interface

• Examination interface

• Document delivery

Retrieval System Model

Query Formulation

Detection

Delivery

Selection

Examination

Index

Docs

User

Indexing

Query Formulation

Query Formulation

Detection

User

Index

Starfield

NLP in IR

The Different Levels of Language Analysis

1-Phonetic or Phonological Level

2-Morphological Level

3-Syntactic Level

4-Semantic Level

5-Discourse Level

How Information Retrieval Works

Step 1: Document Processing

Step 2: Query Processing

Step 3: Query Matching

Step 4: Ranking & Sorting

Intelligent Information Retrieval

orKnowledge Based IR

What Is Different From IR?

• IR is more concerned with words and concepts.• IIR or KBIR is more concerned about relations.• Most of IR models assume term independence.• IIR or KBIR acknowledges existence of

relationships.• IR more suitable for large scale and general retrieval• IIR or KBIR more suitable for domain specific tasks.

Knowledge Based IR

IIR-KBIR

• Expectation or Interaction With User

• Objects

• KB

• Relation Between the objects

• Reasoning

• Learning

• Relation Extraction

Experiments in Farsi Retrieval

Retrieval Models Investigated

• Fuzzy Logic– MMM, Paice

• Vector Space

• Probabilistic, BM25

• N-Grams

• Combinational