1

Chinese Commonsense Processing

Presented by Yen-Ling Kuo

2009/3/30

2

Remember how to build ConceptNet 2…?

• The data we collected:

聖誕節 吃大餐__ 的時候，你會 __

(2, 1, 0)

frequencyfrequency good rankgood rank bad rankbad rank

relationrelation subjectsubjectsubjectsubject

3

Dirty Words Filter

• Idea: Spam filter– Machine learning– Matching/Fuzzy hashing– Black list

• Attribute selection– 近朱者赤，近墨者黑

– Node degree– Ratio of bad rank– # rank– # neighbors in black list– Distance to black list– Confidence of users

Black list White list

Attribute Selection

Classification

Bad subjects

Subjects

Good subjects

4

Link Prediction

• Idea: Social network link prediction

• Application of social network link predication– In Question-Answering Bulletin Board(QABB): 1. Recommend potential answers based on previous communications 2. Predict future hot questions

5

Link Prediction Methods

•Node attribute: not always available• Structural property

Node based topological pattern Node based topological pattern

Path based topological pattern Path based topological pattern

6

Link prediction in Chinese Commonsense?

•Use both node attribute and structural property → Modeled as a supervised learning problem

– Give weights to links according to frequency and good/bad ranks.

– Node attribute

詞類 , relation types

– Structural property

Distance, Weighted common neighbors,Weighted Adamic/Adar,Types of neighbors,Katz

Weighted common neighbors

7

Context Finding

•Determine the context around a concept is useful for building applications.•Context finding is similar to memory search. → Use spreading activation from a source node to get the contextual neighborhood.

Ask 睡覺

枕頭

刷牙浴室

11

0.10.1

0.10.1

0.60.6

0.60.6

0.20.2

0.120.12

※ Different relation

with 　 different weight.

8

Conceptual Analogy

• Employ structure-mapping methods to get a list of structurally analogous concepts given a source concept.

• Structural analogy is not just a measure of semantic distance, ex. “wedding” and “bride”.

貓

貓 有 __

貓 會 __

貓 喜歡 __ 狗

狗 有 __

狗 會 __

狗 喜歡 __

貓 and 狗 are conceptually analogous concepts.

9

Reference

•Hugo Liu, Push Sing. Commonsense Reasoning in and over Natural Language, Lecture Notes in Computer Science, 2004.

•David Liben-Nowell, Jon Kleinberg. The Link Prediction Problem for Social Network, Proceedings of CIKM, 2003.

• Tsuyoshi Murata and Sakiko Moriyasu. Link prediction of social networks based on weighted proximity measures, International Conference on Web Intelligence, 2007