csa4080: adaptive hypertext systems ii

1 of [email protected] University of Malta

CSA4080: Topic 6© 2004- Chris Staff

CSA4080:Adaptive Hypertext Systems II

Dr. Christopher StaffDepartment of Computer Science & AI

University of Malta

Topic 6: Information and Knowledge Representation



Aims and Objectives

• Models of Information Retrieval– Vector Space Model– Probabilistic Model

• Relevance Feedback

• Query Reformulation



Aims and Objectives

• Dealing with General Knowledge

• Programs that reason

• Conceptual Graphs

• Intelligent Tutoring Systems



Background

• We’ve talked about how user information can be represented

• We need to be able to represent information about the domain so that we can reason about what the user’s interests are, etc.

• We covered the difference between data, information, and knowledge in CSA3080...



Background

• In 1945, Vannevar Bush writes “As We May Think”– Gives rise to seeking “intelligent” solutions to

information retrieval, etc.

• In 1949, Warren Weaver writes that if Chinese is English + codification, then machine translation should be possible– Leads to surface-based/statistical techniques



Background

• Even today, nearly 60 years later, there is significant effort in both directions

• For years, intelligent solutions were hampered by the lack of fast enough hardware, software– Doesn’t seem to be an issue any longer, and the

Semantic Web may be testimony to that– But there are sceptics



Background

• Take IR as an example

• At the dumb end we have “reasonable” generic systems, but at other end, systems are domain specific, more expensive, but do they give “better” results?



Background

• At what point does it cease to be cost effective to attempt more intelligent solutions to the IR problem?



Background

• Is “Information” Retrieval a misnomer?– Consider your favourite Web-based IR

system... does it retrieve information?– Can you ask “Find me information about all

flights between Malta and London”?• And what would you get back?

– Can you ask “Who was the first man on the moon?”



Background

• With many IR systems that we use, the “intelligence” is firmly rooted in the user– We must learn how to construct our queries so

that we get the information we seek– We sift through relevant and non-relevant

documents in the results list

• What we can hope for is that “patterns” can be identified to make life easier for us - e.g., recommender systems



Background

• Surface-based techniques tend to look for and re-use patterns as heuristics, without attempting to encode “meaning”

• The Semantic Web, and other “intelligent” approaches, try to encode meaning so that it can be reasoned with and about

• Cynics/sceptics/opponents believe that there is more success to be had in giving users more support, than to encode meaning into documents to support automation



However...

• We will cover both surface-based and some knowledge-based approaches to supporting the user in his or her task



Information Retrieval

• We will discuss two IR models...– Vector Space Model– Probabilistic Model

• ... and surface-based techniques that can improve their usability– Relevance Feedback– Query Reformulation– Question-Answering



Knowledge

• Conceptual graphs support the encoding and matching of concepts

• Conceptual graphs are more “intelligent” and can be used to overcome some problems like the Vocabulary Problem



Reasoning on the Web

• REWERSE (FP6 NoE) is an attempt to represent meaning contained in documents and to reason with and about it so that a single high-level user request may be carried out even if it contains several sub-tasks– E.g., “Find me information about cheap flights

between Malta and London”



Vector-Space Model

• Recommended Reading– p18-wong (Generalised Vector Space

Model).pdf - look at refs [1],[2],[3] for original work



Vector-Space Model

• Documents are represented as m-dimensional vectors or “bags of words”

• m is the size of the vocabulary

• wk = 1, indicates term is present in document

• wk = 0, indicates term is absent

• dj = <1,0,0,1,...,0,0>



Vector-Space Model



Vector-Space Model

• The query is then plotted into m-dimensional space and the nearest neighbours are the most relevant

• However, the results set is usually presented as a list ranked by similarity to the query



Vector-Space Model

• Cosine Similarity Measure (from IR vector space model.pdf)

€

sim(Q,D) =

wqk • wdk

k=1

m

∑

(wqk )2 •k=1

m

∑ (wdk )2

k=1

m

∑



Vector-Space Model

• Calculating term weights– Term weights may be binary, integers, or reals– Binary values are thresholded, rather than

simply indicating presence or absence– Integers or reals will be measure of relative

significance of term in document

• Usually, term weight is TFxIDF



Vector-Space Model

• Steps in calculating term weights– Remove stop words– Stem remaining words– Count term frequency (TF)– Count number of documents containing term

(DF)– Invert it (log(C/DF)), where C is total number

of documents in collection



Vector-Space Model

• Normalising weights for vector length– Documents with longer vectors have a better

chance of being retrieved than short ones (simply because there are a larger number of terms that they will match in a query)

– IR should treat all relevant documents as important for retrieval purposes

– Solution: , where w is weight of term t

€

w = w (wi)2

vector(i)

∑



Vector-Space Model

• Why does this work?– Term discrimination– Assumes that terms with high TF and low DF

are good discriminators of relevant documents– Because documents are ranked, documents do

not need to contain precisely the terms expressed in the query

– We cannot say anything (in VSM) about terms that occur in relevant and non-relevant documents - though we can in probabilistic IR



Vector-Space Model

• Vector-Space Model is also used by Recommender Systems to index user profiles and product, or item, features

• Apart from ranking documents, results lists can be controlled (to list top n relevant documents), and query can be automatically reformulated based on relevance feedback



Relevance Feedback

• When a user is shown a list of retrieved documents, user can give relevance judgements

• System can take original query and relevance judgements and re-compute the query

• Rocchio...



Relevance Feedback

• Basic Assumptions –Similar docs are near each other in vector space–Starting from some initial query, the query can

be reformulated to reflect subjective relevance judgements given by the user

–By reformulating the query we can move the query closer to more relevant docs and further away from nonrelevant docs



Relevance Feedback

• In VSM, reformulating query means re-weighting terms in query

• Not failsafe: may move query towards nonrelevant docs!



Relevance Feedback

• The Ideal Query

• If we know the answer set rel, then the ideal query is:

€

Qopt =1

RDi −

1

N − RDi

Di ∈Nonrel

∑Di ∈rel

∑



Relevance Feedback

• In reality, a typical interaction will be:– User formulates query and submits it– IR system retrieves set of documents– User selects R’ and N’

where 0 <= <= 1 (and vector magnitude usually dropped...)

€

Q i+1 = αQ i +β

′ R Di −

γ

′ N Di

Di ∈ ′ N

∑Di ∈ ′ R

∑

€

Q i+1 = αQ i + β Di − γ DiDi ∈ ′ N

∑Di ∈ ′ R

∑



Relevance Feedback

• What are the values of and ?– is typically given a value of 0.75, but this

can vary. Also, after a number of iterations, the original weights of terms can be highly reduced

– If and have equal weight, then relevant and nonrelevant docs make equal contribution to reformulated query

– If = 1, = 0, then only relevant docs are used in reformulated query

– Usually, use = 0.75, = 0.25



Relevance Feedback

• ExampleQ: (5, 0, 3, 0, 1)R: (2, 1, 2, 0, 0) N: (1, 0, 0, 0, 2) = 0.75, = 0.50, = 0.25Q’ = 0.75Q + 0.5R – 0.25N

= 0.75(5, 0, 3, 0, 1)+0.5(2, 1, 2, 0, 0)–0.25(1,0, 0, 0, 2)

= (4.5, 0.5, 3.25, 0, 0.25)



Relevance Feedback

• How many docs to use in R’ and N’?– Use all docs selected by user– Use all rel docs and highest ranking nonrel docs– Usually, user selects only relevant docs...

• Should entire document vector be used?– Really want to identify the significant terms...

• Use terms with high-frequency/weight• Use terms in doc adjacent to terms from query• Use only common terms in R’ (and N’)



Automatic Relevance Feedback

• Users tend not to select nonrelevant documents, and rarely choose more than one relevant document (http://www.dlib.org/dlib

/november95/11croft.html)– This makes it difficult to use relevance

feedback

• Current research uses automatic relevance feedback techniques...




• Two main approaches– To improve precision– To improve recall




• Reasons for low precision– Documents contain query terms, but documents

are not “about” the “concept” or “topic” the user is interested in

– E.g., user wants documents in which a cat chases a dog but the query <cat, chase, dog> also retrieves docs in which dogs chase cats

– Term ambiguity




• Improving precision– Want to promote relevant documents in the

results list– Assume that top-n (typically 20) documents

are relevant, and assume docs ranked 500-1000 are nonrelevant

– Choose co-occurring discriminatory terms– Re-rank docs ranked 21-499 using (modified)

Rocchio methodp206-mitra.pdf




• Improving precision– Does improve precision by 6%-13% at p-21 to

p-100– But remember that precision is to do with the

ratio of relevant to nonrelevant documents retrieved

– There may be many relevant documents that were never retrieved (i.e., low recall)




• Reasons for low recall– “Concept” or “topic” that user is interested in

can be described using terms additional to those express by user in query

– E.g., think of all the different ways in which you can express “car”, including manufacturers names (e.g., Ford, Vauxhall, etc.)

– There is only a small probability that user and author use the same term to describe the same concept




• Reasons for low recall– “Imprudent” query term “expansion” improves

recall, simply because more documents are retrieved, but hurts precision!




• Improving recall– Manually or automatically generated thesaurus

used to expand query terms before query is submitted

– We’re currently working on other techniques to pick synonyms that are likely to be relevant

– Semantic Web attempts to encode semantic meaning into documents

p61-voorhees.pdf, qiu94improving.pdf, MandalaSigir99EvComboWordNet.pdf



Indexing Documents

• Obviously, comparing a query vector to each document vector to determine the similarity is expensive

• So how can we do it efficiently, especially for gigantic document collections, like the Web?



Indexing Documents

• Inverted indices– An inverted index is a list of terms in the

vocabulary together with a postings list for each term

– A postings list is a list of documents containing the term



Indexing Documents

• Inverted index

• Several pieces of information can be stored in the postings list– term weight

– location of the term in the document (to support proximity operators)



Indexing Documents

• Results set is obtained using set operators• Once documents in results set are known,

their vectors can be retrieved to perform ranking operations on them

• The document vectors also allow automatic query reformulation to occur following relevance feedback

• See brin.pdf and p2-arasu.pdf



Probabilistic IR

• VSM assumes that a document that contains some term x is about that term

• PIR compares the probability of seeing term x in a relevant document as opposed to a nonrelevant document

• Binary Independence Retrieval Model proposed by Robertson & Sparck Jones, 1976

robertson97simple.pdf, SparckJones98.pdf



BIR

• BIR Fundamentals:– Given a user query there is a set of documents

which contains exactly the relevant documents and no other:

• the “ideal” answer set– Given the ideal answer set, a query can be

constructed that retrieves exactly this set• Assumes that relevant documents are

“clustered”, and that terms used adequately discriminate against non-relevant documents



BIR

• We do not know what are, in general, the properties of the ideal answer set

• All we know is that documents have terms which “capture” semantic meaning

• When user submits a query, “guess” what might be the ideal answer set

• Allow user to interact, to describe the probabilistic description of the ideal answer set (by marking docs as relevant/non-relevant)



BIR

• Probabilistic Principle: Assumption– Given a user query q and a document dj in the

collection:• Estimate the probability that the user will find dj

relevant to q

– Rank documents in order of their probability of relevance to the query (Probability Ranking Principle)



BIR

• Model assumes that probability of relevance depends on q and doc representations only

• Assumes that there is an ideal answer set!

• Assumes that terms are distributed differently in relevant and non-relevant documents



BIR

• Whether or not a document x is retrieved depends on:– Pr(rel|x): the probability that x is relevant– Pr(nonrel|x): ... that x isn’t relevant



BIR

• Document Ranking Function: document x will be retrieved if

where a2 is the cost of not retrieving a relevant document, and a1 is the cost of retrieving a non-relevant document

• If we knew Pr(rel|x) (or Pr(nonrel|x)), solution would be trivial, but...

€

a2 Pr(rel | x) ≥ a1 Pr(nonrel | x)



BIR

• Use Bayes Theorem to rewrite Pr(rel|x):

– Pr(x): probability of observing x– P(rel): a priori probability of relevance (ie,

probability of observing a set of relevant documents)

– : probability that x is in the given set of relevant docs

€

Pr(rel | x) =Pr(x | rel)P(rel)

Pr(x)

€

Pr(x | rel)



BIR

• Can do the same for

€

Pr(nonrel | x)

€

Pr(nonrel | x) =Pr(x | nonrel)P(nonrel)

Pr(x)



BIR

• The document ranking function can be rewritten as:

• and simplified as:

• Pr(x | rel) and Pr(x | nonrel) are still unknown, so we will replace them in terms of keywords in the document!€

log g(x) = logPr(x | rel)

Pr(x | nonrel)+

Pr(rel)

Pr(nonrel)€

log g(x) = logPr(x | rel)Pr(rel)

Pr(x)

Pr(x)

Pr(x | nonrel)Pr(nonrel)



BIR

• We assume that terms occur independently in relevant and non-relevant docs...

• : probability that term xi is present in a document randomly selected from the ideal answer set

• : probability that term xi is present in a document randomly selected from outside the ideal answer set

€

log g(x) = logPr(xi | rel)

Pr(xi | nonrel)+C

i =1

t

∑

€

Pr(xi | rel)

€

Pr(xi | nonrel)



BIR

• Considering document , where di is the weight of term i,

• where is the probability that a relevant document contains term xi (similarly for )

€

D = d1,d2 ,K dt

€

logg(x) =Pr(x i = di | rel)

Pr(x i = di | nonrel)+ C

i=1

t

∑

€

Pr(xi = di | rel)

€

Pr(xi = di | nonrel)



BIR

• When di = 0 we want the contribution of term i to g(x) to be 0:

=

€

log g(x) = logPr(xi = di | rel)

Pr(xi = di | nonrel)

Pr(xi = 0 | nonrel)

Pr(xi = 0 | rel)+C

i =1

t

∑

€

log g(x) = logpi(1− qi )

qi(1− pi )+C

i =1

t

∑



BIR

• The term relevance weight of term xi is:

• Weight of term i in document j is:€

tri =

logpi(1− qi)

qi(1− pi)=

logPr(x i = di | rel)

Pr(x i = di | nonrel)

Pr(x i = 0 | nonrel)

Pr(x i = 0 | rel)

€

wij = tfij × tri



BIR

• Estimation of term occurrence probability– Given a query, a document collection can be

partitioned into a relevant and non-relevant set– The importance of a term j is its discriminatory

power in distinguishing between relevant and nonrelevant documents



BIRWith complete information about the relevant & non-relevant document sets we can estimate pj and qj:

• Approximation:

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

€

pj = rj R

€

qj =dfj − rj

N − R

€

trj = logrj R

(dfj − rj) ( N − R)

1− (dfj − rj) ( N − R)

1− rj R

= logrj

R − rj

N − dfj − R + rj

dfj − rj

€

trj = logrj

R

N

dfj



BIR

• Term Occurrence Probability Without Relevance Information– What do we do because we don’t know rj?

– : since most docs are nonrelevant

– pj = 0.5 (arbitrary)

– : does this remind you of anything?

€

qj = dfj N

€

trj = log(( N dfj )−1)



BIR

• Reminder... Ranking Function

where,

pi = Pr(xi=di|rel)

qi = Pr(xi=di|nonrel)and di is the weight of term i

€

log g(x) = logpi(1− qi )

qi(1− pi )+C

i =1

t

∑



Relevance Feedback in BIR

• Want to add more terms to the query so the query will resemble documents marked as relevant (note difference from VSM)

• How do we select which terms to add to the query?



Relevance Feedback in BIR

• Rank terms in marked documents and add the first m terms

• where:

N: no. of docs in the collectionni: document frequency of term iR: no. of relevant docs selectedri: no. of docs in R containing term i

• Compares frequency of occurrence of term in R with document frequency

€

wi = logri N − ni − R + ri( )

R − ri( ) ni − ri( )



Question-Answering on the Web

• Two aspects to IR:– Coverage (find all relevant documents)– Question-Answering (find the answer to

specific query)

• In QA we want one answer to our question• How much NLP do we need to use to

answer fact-based questions– Answers that require reasoning are much

harder!



Question Answering

• Most IR tasks assume that user can predict what terms a relevant document will contain

• But sometimes what we want is the answer to a direct question– “Who was the first man on the moon?”

• Do we really want a list of millions of documents that contain first, man, moon?

• And do we really want to have to read them to find the answer?



Question Answering

• All we want is one document, or one statement, that contains the answer

• Can we take advantage of IR on the Web to do this?

• Taking advantage of redundancy on the Web– E.g., Mulder, Dumais



Mulder

• Uses Web as collection of answers to factual questions– “Who was the first man on the moon?”– “What is the capital of Italy?”– “Where is the Taj Mahal?”

kwok01scaling.pdf



Mulder

• Three parts to a QA system:– Retrieval Engine

• Indexes documents in a collection and retrieves them

– Query Formulator• Converts NL question into formal query

– Answer Extractor• Locates answer in text



Mulder

• Six parts to Mulder:– Question Parsing– Question Classification– Query Formulation– Search Engine– Answer Extraction– Answer Selection



Dumais et al

• Takes advantage of multiple, differently phrased, answer occurrences on Web

• Doesn’t need to find all answer phrases– Just the ones that match the query pattern

• Rules for converting questions, finding answers are mostly handwritten

p291-dumais



Dumais et al

• Steps– Rewrite question into weighted query patterns

• Use POS tagger + lexicon to seek alternative word forms

– Search– Mine N-grams in summaries– Filter and re-weight N-grams– Tile N-grams to yield longer answers



Azzopardi

• Joel Azzopardi, 2004, “Template-Based Fact Finding on the Web” FYP report, CSAI

• Can find factoids about a series of queries relating to a particular topic using majority polling (voting) to decide amongst competing answers

• Series of topic sensitive query patterns stored in template



Azzopardi

• Template is learned by comparing a sample of documents about a topic

• Commonly occurring phrases (trigrams) extracted and turned into partial query in template, together with answer “type”



Azzopardi

• When user wants information regarding a topic, use appropriate template together with subject (e.g., person’s name)

• Subject is appended to partial queries in template - queries are submitted to Google

• Top-n documents retrieved and processed to identify candidate answers

• Uses voting to decide on most frequently occurring answer



Summary

• We’ve discussed a couple of popular models of IR that are more “intelligent” that plain old Extended Boolean Information Retrieval

• They still treat terms as atoms that are representative of the semantic meaning of the document



Summary

• But word order generally insignificant (“bag of words”)– Cannot distinguish between “dog chased cat”

and “cat chased dog” • unless phrase matching also used, but then cannot tell

that “cat chased dog” and “dog was chased by cat” are semantically equivalent

• What about information extraction?– George W. Bush = President of the United States

of America



Summary

• More “intelligent” approaches have been used

• And more “intelligence” is being put “into” the Web

• Personalisation and user-adaptivity also require high accuracy in determining which documents are relevant to a user



Summary

• Sowa’s conceptual graphs and McCarthy’s Generality in AI/Notes on Contextual Reasoning are seminal works that underpin much that is happening in the Semantic Web

• CGs represent semantic content of utterances in interchangeable format (KIF)

• McCarthy claims that it is hard to make correct inferences in the absence of contextual information



Summary

• Because of the expense of CGs, they are still very much domain specific– SemWeb hopes that by bringing massive

numbers of people together there will be a proliferation of “ontologies” to make it happen

• Guha did his PhD “Contexts: A Formalisation and Some Applications” at Stanford, under John McCarthy. His work on Cyc underpins RDF, DAML+OIL



Dealing with General Knowledge

• Why did “Mary hit the piggy bank with a hammer”?




• Do computer systems need general knowledge?

• How do computer systems represent general knowledge?




• Do we need general knowledge?

• How do we represent general knowledge?




• As usual, has its roots in philosophy (epistemology)– Early (i.e., Greek) revolved around Absolute

and Universal Ideas and Forms (Plato)– Aristotle: Logic for representing and reasoning

about knowledge

http://pespmc1.vub.ac.be/EPISTEMI.html




• Following Renaissance, two main schools of thought

• Empiricists– Knowledge as product of sensory perception

• Rationalists– Product of rational reflection




• Kantian Synthesis of empiricism and reflectionism– Knowledge results from the organization of

perceptual data on the basis of inborn cognitive structures, called "categories".

– Categories include space, time, objects and causality.

– (viz. Chomsky’s Universal Grammar)




• Pragmatism– Knowledge consists of models that attempt to

represent the environment to simplify problem-solving

– Assumption: Models are “rich”. No model can ever hope to capture all relevant information, and even if such a complete model would exist, it would be too complicated to use in any practical way.




• Pragmatism (contd.)– The model which is to be chosen depends on the

problems that are to be solved (context).• But see also discussions on pragmatic vs. cognitive

contexts! (Topic 3)

– Basic criterion: model should produce correct (or approximate) (testable) predictions or problem-solutions, and be as simple as possible.

• This is the approach mainly used in CS/AI today




• “The first theories of knowledge stressed its absolute, permanent character, whereas the later theories put the emphasis on its relativity or situation-dependence, its continuous development or evolution, and its active interference with the world and its subjects and objects. The whole trend moves from a static, passive view of knowledge towards a more and more adaptive and active one”.

http://pespmc1.vub.ac.be/EPISTEMI.html




• We’ll look at four overviews of and approaches to knowledge in computer systems– McCarthy (1959, mcc.pdf) – Sowa (1979, p79-1010.pdf)– McCarthy (1987, p1030-mccarthy.pdf)– Brézillon & Pomerol (2001, is-context-a-kind.pdf)




• McCarthy, J. 1959. “Programs with Common Sense”

• “a program has common sense if it automatically deduces for itself a sufficiently wide class of immediate consequences of anything it is told and what it already knows”.




• Objective: “to make programs that learn from their experience as effectively as humans do”

• To learn to improve how to learn

• And to do it in logic using a logical representation




• Minimum features required of a machine that can evolve intelligence approaching that of humans– Representation of all behaviours– Interesting changes in behaviour must be

expressible– All aspects of behaviour must be improvable– Must have notion of partial success– System must be able to create/learn subroutines




• Bar-Hillel’s biggest complaint (in my opinion) is– “A deductive argument, where you have first to

find out what are the relevant premises, is something which many humans are not always able to carry out successfully. I do not see the slightest reason to believe that at present machines should be able to perform things that humans find trouble in doing”

– We’ll return to this in Closed vs. Open World Assumption




• Sowa, J. 1979. “Semantics of Conceptual Graphs”

• Logic used by McCarthy as representation of “statements about the world” as well as “theorem prover” to infer/deduce new knowledge (assumptions) about the world

• Sowa uses CG as “a language for representing knowledge and patterns for constructing models”




• Sowa proposes CGs as better alternative to semantic networks and predicate calculus– SemNets have no well-defined semantics– PC is “adequate for describing mathematical

theories with a closed set of axioms... But the real world is messy, incompletely explored, and full of unexpected surprises”




• CGs serve two purposes:– They can be used as canonical representations

of meaning in Natural Language– They can be used to construct abstract

structures that serve as models in the model-theoretic sense (e.g., microtheories)




• To understand a sentence:1. Convert utterance to CG2. Join CG to graphs that help resolve

ambiguities and incorporate background information

3. Resulting graph is nucleus for constructing models (of worlds) in which utterance is true

4. Laws of world block illegal extensions5. If model could be extended infinitely, result

would be complete standard model




• “Mary hit the piggy bank with a hammer”




• Linearizing the conceptual graph

[PERSON:Mary]->(AGNT)->[HIT:c1]<-(INST) <- [HAMMER]

[HIT:c1]<-(PTNT)<-[PIGGY-BANK:i22103]




• Context-sensitive logical operators– Allow building models of possible worlds and

checking their consistency– Def: “A sequent is a collection of conceptual

graphs divided into two sets, called the conditions u1,..., un and the assertions v1,..., vm. It is written u1,..., un ->v1,..., vm.”




• Cases of sequents:– simple assertion: no conditions, one assertion (->v)– disjunction: no conditions, one or more assertions

(->v1,..., vm)– simple denial: one condition, no assertions (u->)– compound denial: 2 or more conditions, no assertions

(u1,..., un->)– conditional assertion: u1,..., un ->v1,..., vm

– empty clause: ->– Horn clause: anything with at most one assertion (inc. 0)




• McCarthy, J. 1987. “Generality in Artificial Intelligence” (1971 Turing Award Lecture)

• “no one knows how to make a general database of commonsense knowledge that could be used by any program that needed the knowledge”

• Examples: robots moving things around, what we know about families, buying and selling...




• “In my opinion, getting a language [my italics] for expressing general commonsense knowledge for inclusion in a general database is the key problem of generality in AI.”




• How can we write programs that can learn to modify their own behaviour, including improving the way they learn?– Friedberg (A Learning Machine, c. 1958)– Newell, Simon, Shaw (General Problem Solver, c. 1957-1969)– Newell, Simon (Production Machines, 1950-1972)– McCarthy (Logical Representation, c. 1958)– McCarthy (Formalising Context, 1987)




• A Learning Machine– Learns by making random modifications to a

program– Discard flawed programs– Learnt to move a bit from one memory cell to

another– In 1987, was demonstrated to be inferior to

simply re-writing the entire program




• General Problem Solver– Represent problems of some class as problems

of transforming one expression into another using a set of allowed rules

– First system to separate problem structure from the domain

– McCarthy claims problem in representing commonsense knowledge as transformations




• Production (Expert) Systems– Represent knowledge as facts and rules– Facts contain no variables or quantifiers– New facts are produced by inference,

observation and user input– Rules are usually coded by programmer/expert– Rules are usually not learnt or generated by

system (but see data mining)




• Logical Representation– Representing information declaratively– Although Prolog can represent facts in logical

representation and reason using logic, it cannot do universal generalization, and so cannot modify its own behaviour enough

– So McCarthy built Lisp...




• Logical Representation– McCarthy’s “dream” is that commonsense

knowledge possessed by humans could be written as logical sentences and stored in a db

– Facts about the effects of actions is essential (when we hear the squeal of types we expect a bang...)

– Necessary to say that an action changes only features of the situation to which it refers




• Context– We understand under-qualified utterances

because we understand them in context– “The book is on the table”– “Where is the book?”




• Context– “Can you fetch me the book, please?”– Up until the last utterance, the physical location

of the book was not significant, and we were able to have a short dialogue about it

– Fully qualified utterances are too unwieldy to use in conversation

– Occasionally gives rise to misunderstandings...




• Context– “The book is on the table” is valid for a large

number of different contexts, in which the specific book and the specific table, and perhaps even the location of the specific table can be significant and can also change over time

– Utterances are understood in context




• Is Context a ... collective Tacit Knowledge?– How does data become knowledge?




• Is Context a ... collective Tacit Knowledge?– Context is “the collection of relevant conditions

and surrounding influences that make a situation unique and comprehensible”




– Where is context?




• Closed world vs. Open World assumption– Closed World

• I assume that anything I don’t know the truth of is false: I know everything that is true

– Open World• I assume that anything I don’t know the truth of is

unknown: Some things I don’t know may be true: I don’t know everything




• Prolog, for instance, will return “false” about any fact that is missing from its database, or for which it cannot derive a truth-value

• A three-valued logic permits assertions to be true, false, or unknown

• However, reasoning and truth-maintenance become expensive in the open world




• The Web is an open world so the Semantic Web needs to reason within an open world (perhaps even across ontologies)

• Doesn’t mean that to solve some problems, SW cannot temporarily assume a closed-world (within an agreed ontology)

ekaw2004.pdf



Teaching Knowledge

• Intelligent Tutoring Systems need to model both the user and the domain to create a learning path based on the students prior knowledge and goals, and to monitor the student’s progress

• AHSs developed partly by using hypertext systems as domain representations for ITSs - basically, when intelligent tutoring moved to the Web



Intelligent Tutoring Systems

• Overview• Modern ITS development began in 1987,

after a review by Wenger– Wenger, E. (1987). Artificial Intelligence and Tutoring

Systems: Computational and Cognitive Approaches to the Communication of Knowledge. Los Altos, CA: Morgan Kaufmann Publishers, Inc.

• This was the first attempt to examine the implicit and explicit goals of ITS designers




• Wenger described ITS as a part of "knowledge communication" and his review focused on cognitive and learning aspects as well as the AI issues




• "... consider again the example of books: they have certainly outperformed people in the precision and permanence of their memory, and the reliability of their patience. For this reason, they have been invaluable to humankind. Now imagine active books that can interact with the reader to communicate knowledge at the appropriate level, selectively highlighting the interconnectedness and ramifications of items, recalling relevant information, probing understanding, explaining difficult areas in more depth, skipping over seemingly known material ... intelligent knowledge communication systems are indeed an attractive dream." (p. 6).




• Motivations underlying ITSs (and education in general):– to teach about something (abstract)– to teach how to do something (practical)




• How can learning be achieved?– By rote– By mimicry (observation)– By application




• When student performs task correctly, assume student understands concept and/or its application– When student performs task incorrectly, how

can the tutor help?– Simply tell the student the correct answer– Tell student the correct answer and state why

it's correct– Explain to the student why his/her answer is

incorrect




• Explanation-based correction is HARD!

• Tutor must first understand why the student gave the incorrect answer– Student lacks knowledge– Incorrect application of correct procedure– Misinterpretation of task– Misconception of principle




• How to tutor?– Originally Computer-Aided Instruction (CAI)

used non-interactive "classroom" techniques.– All students were taught in the same manner

(e.g., through flash cards) and then assessed. – If a student failed, student had to work through

the same material again, to "learn it better"– Access to human tutor to address difficulties– This type of learning, although self-paced, is

ineffective




• The goal of an ITS– A student learns from ITS by solving problems. – The ITS selects a problem and compares its

solution with that of the student– It performs a diagnosis based on the

differences. – After giving feedback, system reassesses and

updates the student skills model and entire cycle is repeated.




• The goal of an ITS (continued):– As the system assesses what the student

knows, it also considers what the student needs to know, which part of the curriculum is to be taught next, and how to present the material.

– It then selects the next problem/s.




Basic issues in knowledge communication




• Domain Expertise– Rather than being represented by chunks of

information, the domain should be represented using a model and a set of rules which allows the system to "reason"

– Typical domain model representations (make closed world assumption!)

• If - Then Rules • If - Then Rules with uncertainty measures • Semantic Networks• Frame based representations




• Student Model– According to Wenger, student models have

three tasks. They must• Gather information about the student (implicitly or

explicitly)• Create a representation of the student's knowledge

and learning process (often as “buggy” models)• Perform a diagnosis to determine what the student

knows and to determine how the student should be taught and to identify misconceptions




• Student model architectures – Overlay student models – Differential student models – Perturbation student models




• Student model diagnosis – Performance measuring – Model tracing – Issue tracing – Expert systems




• Pedagogical expertise– Used to decide how to:

• present/sequence information • answer questions/give explanations• provide help/guidance/remediation




• According to Wenger, when "learning is viewed as successive transitions between knowledge states, the purpose of teaching is accordingly to facilitate the student's traversal of the space of knowledge states." (p. 365)

• The ITS must model the student's current knowledge and support the transition to a new knowledge state.




• ITSs must alternate between diagnostic and didactic support.

• Diagnostic support– Information about a student's state is inferred

on 3 levels– Behavioural - ignores learner's knowledge, and

concentrates on observed behaviour– Epistemic - attempts to infer learner's

knowledge state based on learner's behaviour– Individual - cognitive model of learner's state,

attitudes (to self, world, ITS), motivation




• Didactic support– Concerned with the "delivery" aspect of

teaching




• Interface– The interface is the layer through which the

learner and ITS communicate– The design of an interface which enhances

learning is essential– Web-based ITSs tend to rely on the Web

browser to provide the interface– Hypermedia-based ITSs in general must provide

adaptive presentation and adaptive navigation facilities, if they are to extend beyond knowledge exploration environments