fuzzy database & information retrieval
Post on 14-Jan-2016
49 Views
Preview:
DESCRIPTION
TRANSCRIPT
Fuzzy Database & Information Retrieval
Similarity relation defined for the domain opinion
18.06.02.0008.018.06.02.006.08.018.06.02.02.06.08.018.06.0
02.06.08.018.0002.06.08.01HNNSNSFFHF
HNNSNSFFHF
Query:which sociologists are in considerable agreement with Kass
concerning policy Y?
Fuzzy Relational Data Base: Buckles, Petry(1) Elements of the tuples contained in the relations may be
subsets of the domain universal set.(2) A similarity relation is defined on each domain universal set.
Fuzzy Data Base1. (Project (select Assessment where Name = Kass and Option =
Y) over Opinion) giving R1
Relation : R1 Option
favorable
Retrieve the opinion of Kass concerning option Y
2. (Project (select Expert where Field = Sociologist) over Name) giving R2
Relation : R2 Name
Osborn
Schreiber
Cohen
Specterman
Select all sociologists from the table of experts
3. (Project (select (Join R2 and Assessment over Name) where Opinion = Y) over Name, Opinion) giving R3
List the opinions of the sociologists
4. (Join R3 and R1 over Opinion) with THRES (Opinion) 0.75 ≧and THRES (Name) 0≧
Relation : R3Name Opinion
Obsorn Slightly favorable
Schreiber Favorable
Cohen Slightly negative
Specterman Highly favorable
Name Opinion
{Obsorn, Schreiber Specterman}
Slightly favorable, favorable, highly favorable}
Information Retrieval
1. IR process can be viewed as a knowledge communication process, which involves learning and problem solving strategies.
2. IR system controls the knowledge flow between documents and the user.
3. An IR system is a computer system that allows users to retrieve information from a document connection stored in a data base.
4. Classical IR is built on Boolean algebra framework.
Fuzzy Information Retrieval
1. Fuzzy relation for the grade of relevance between index terms and documents:
– R: X x Y [0,1]– Determined subjectively or objectively– Number of occurrences; publication dates,
document types
2. Fuzzy thesaurus– T: X x X [0,1]
Fuzzy Information Retrieval
1. Information retrieval model
2. Advantages:– R and T are more expressive and their
construction is more realistic– Fuzzy inquiry provides greater flexibility
RBD
xxTxAxB
TAB
jiiXx
ji
)],(,min[ max
Information retrieval based on fuzzy associations
1. Introduction
2. Three components in information retrieval
3. Fuzziness in a thesaurus: first component
4. Fuzziness in retrieval: second component
5. Fuzziness on output: third component
6. Classification of output
7. Conclusion
2. Three components in information retrieval D = {d1,d2,…,dn} be a finite set of documents for
retrieval W = {w1,w2,…,wm} denote a set of descriptors T : D ─>[0,1]w . T(d): a subset of descriptors in W
indexed to the document d. U(U = T-1) . U(w): documents have keyword w.
F
Information retrieval based on fuzzy associations
U P rr’
q
3. Fuzziness in a thesaurus: first componentThree type thesaurus (represented as binary relation)RT: related termsNT: narrower termsBT: broader termsB(v,w) = N(w,v) R(v,w) = R(w,v)Method of automatic generation of thesauri:1. Typical:counting frequencies of simultaneous occurrences
of pairs of keywords in a set of documents.2. Fuzzy set model:C = {c1,c2,…,cp} be a finite set of concepts where each ci, i=1,
…p represents a unit of conceptH:W ─>[0,1]p a fuzzy set valued function which maps each
keyword to it’s corresponding concepts as a fuzzy set in C.Wwwh :)( is concept of the word w.
)(
)()(),(
)()(
)()(),(
vh
whvhwvN
whvh
whvhwvR
Even by present computers, it’s difficult to calculate values of the fuzzy relation above using array in straightforward way, since the numbers of elements in W and D are very large(103 x 105). Although techniques to handle sparse matrices may be applied, there is another method for generation R and N based on manipulation of sequential files. The principle tool for this is sorting.
(a,b,c) means a record in which field are a,b and c.{(a,b,c)} means a set of records such as (a,b,c).
Input: a set D of documents, Each document d ∈ D has a number of keywords in W.A keyword may occur twice or more in a document. The frequency of occurrence of wi in dk is denoted by hik.
Output: a set of records {(wi,wj,R(wi,wj)]} for all pairs R(wi,wj)<>0
Algorithm GFT (generation of a fuzzy thesaurus).// Find pairs of keywords in every document.//For all dk D do∈ find all keywords wi W and calculate h∈ jk
for all (wi,wj),wi<wj, that are found in dk domake record (wi,wj, min(hik,hjk))output (wi,wj,min(hik,hjk) to WORK1
repeat for all wi that are found in dk do
make record (wi, hjk)output (wi,hjk) to WORK2
repeatrepeat//sort WORK1 and WORK2.//sort WORK1 into increasing order of the key (wi,wj)sort WORK2 into increasing order of the key wi
//Calculate R.Scan WORK1 and WORK2.//for all (wi,wj) in WORK1 do find all record for (wi,wj) in WORK1
and all records for wi, and wj in WORK2 R (wi,wj)←∑k min(hik,hjk)/(∑k hik+ ∑k hjk- ∑kmin(hik,hjk)) output (wi,wj,R(wi,wj)) to an output filerepeatend-of algorithm GFT
In a foregoing paper an experimental calculation on three thousand documents and thirty thousand keywords was carried out using GFT based on sorting shows a reasonable amount of 800 sec of CPU time.
//record (di,pi)////before another record (di,pi) satisfies either di<dj or////di = dj, pi > pj//Take the first record (d1,p1) in work(D,P)<-(d1,p1)for all dj in WORK do//the dj’s are sequentially examined.// if D <> dj then
output (D,P) to to an output file OUT(D,P)<-(di,pj)
endifrepeatoutput(D,P) to OUT//OUT contains exactly those records that represent P=Uf(d,w) define by above//
//Third step: if necessary sort again.//sort OUT into the decreasing order of the key p and print OUT
4. Fuzziness in retrieval: second component
For the crisp case a retrieval through a thesaurus
given a keyword w is as follows.
(a) Examine the thesaurus F and find all associated
terms v11,v12,…,v1p.
(b)Find subsets U(v11),U(v12),…,U(v1p).(c) Establish the retrieved set of documents as the
union of U(v11),U(v12),…,U(v1p): ∪1 i p≦ ≦ U(v1i)
Uf(d,w) = 1 iff d U(v∈ 1) for some v1 such that
F(v1,w) = 1,
0 otherwise.
When the thesaurus F is fuzzy and U is crispUf(d,w) = max v W∈ min [U(d,v),F(v,w)].This equation is valid also for a fuzzy relation U(d,v).Algorithm FR(Fuzzy Retrieval).//First step: Find all records.//for all v such that F(v,w) <> 0 in FT do for all d U(v) do∈
p(d,v)<-min[U(d,v),F(v,w)]output record (d,p(d,v)) to a work file WORK
repeatrepeat//second step: Find values of Uf.//sort WORK into increasing order of the first key d and into decreasing order of the second key p//the above sorting means that in the resulting sequence,a//
End-of FR5. Fuzziness on output: third component
Fuzzy filter. EX:(a) Find recent documents that have keyword w.(b) Find documents that have keywords w and are
relevant to one’s field of interestr = r’ ∩ g
6. Classification of output1) Decreasing of membership2) Divide into layers
7. ConclusionProblem for further studies1) Discussion of crisp techniques of advanced
indexing and retrieval using a fuzzy set model,
2) Studies of efficient algorithms for large scale database. In particular, development of hardware for information retrieval should be taken into account.
3) Application of methods in fuzzy information retrieval to related areas.
個人化服務元件技術之研究
郭耀煌教授成功大學資訊工程系
成功大學數位生活科技研究中心
top related