advance information retrieval topics
DESCRIPTION
Advance Information Retrieval Topics. Hassan Bashiri. Information Filtering. Agenda. Information filtering Automatic profile learning Social filtering Training Strategies. Information Access Problems. Different Each Time. Retrieval. Information Need. Data Mining. Filtering. Stable. - PowerPoint PPT PresentationTRANSCRIPT
Advance Information Retrieval Topics
Hassan Bashiri
Information Filtering
Agenda
• Information filtering
• Automatic profile learning
• Social filtering
• Training Strategies
Information Access Problems
Collection
Info
rmat
ion
Nee
d
Stable
Stable
DifferentEach Time
DataMining
Retrieval
Filtering
DifferentEach Time
Indexing and Complexity
Agenda
• Inverted indexes
• Computational complexity
An Example
quick
brown
fox
over
lazy
dog
back
now
time
all
good
men
come
jump
aid
their
party
00110000010010110
01001001001100001
Term Doc
1
Doc
2
00110110110010100
11001001001000001
Doc
3D
oc 4
00010110010010010
01001001000101001
Doc
5D
oc 6
00110010010010010
10001001001111000
Doc
7D
oc 8
A
B
C
FD
GJLMNOPQ
T
AIALBABR
THTI
4 ,82 ,4 ,61 ,3 ,7
1 ,3 ,5 ,72 ,4 ,6 ,8
3 ,53 ,5 ,7
2 ,4 ,6 ,83
1 ,3 ,5 ,72 ,4 ,82 ,6 ,8
1 ,3 ,5 ,7 ,86 ,81 ,31 ,5 ,72 ,4 ,6
PostingsInverted File
The Finished Product
quick
brown
fox
over
lazy
dog
back
now
time
all
good
men
come
jump
aid
their
party
Term
A
B
C
FD
GJLMNOPQ
T
AIALBABR
THTI
4 ,82 ,4 ,61 ,3 ,7
1 ,3 ,5 ,72 ,4 ,6 ,8
3 ,53 ,5 ,7
2 ,4 ,6 ,83
1 ,3 ,5 ,72 ,4 ,82 ,6 ,8
1 ,3 ,5 ,7 ,86 ,81 ,31 ,5 ,72 ,4 ,6
PostingsInverted File
Cross-Language Information Retrieval
Agenda
• Cross-language IR– Controlled vocabulary
• Automatic indexing
– Free text– Evaluation– User interface design
11
What is CLIR?
Users enter their query in one language and the search engine retrieves relevant documents in other languages.
Retrieval SystemEnglish Query French Documents
11
Term-aligned Sentence-aligned Document-aligned Unaligned
Parallel Comparable
Knowledge-based Corpus-based
Controlled Vocabulary Free Text
Cross-Language Text Retrieval
Query Translation Document Translation
Text Translation Vector Translation
Ontology-based Dictionary-based
Thesaurus-based
Retrieval System Interfaces
Agenda
• Query interface
• Selection interface
• Examination interface
• Document delivery
Retrieval System Model
Query Formulation
Detection
Delivery
Selection
Examination
Index
Docs
User
Indexing
Query Formulation
Query Formulation
Detection
User
Index
Starfield
NLP in IR
The Different Levels of Language Analysis
1-Phonetic or Phonological Level
2-Morphological Level
3-Syntactic Level
4-Semantic Level
5-Discourse Level
How Information Retrieval Works
Step 1: Document Processing
Step 2: Query Processing
Step 3: Query Matching
Step 4: Ranking & Sorting
Intelligent Information Retrieval
orKnowledge Based IR
What Is Different From IR?
• IR is more concerned with words and concepts.• IIR or KBIR is more concerned about relations.• Most of IR models assume term independence.• IIR or KBIR acknowledges existence of
relationships.• IR more suitable for large scale and general retrieval• IIR or KBIR more suitable for domain specific tasks.
Knowledge Based IR
IIR-KBIR
• Expectation or Interaction With User
• Objects
• KB
• Relation Between the objects
• Reasoning
• Learning
• Relation Extraction
Experiments in Farsi Retrieval
Retrieval Models Investigated
• Fuzzy Logic– MMM, Paice
• Vector Space
• Probabilistic, BM25
• N-Grams
• Combinational