knowledge acquisition ii. protocol analysis knowledge provider works through some pre-defined task,...
TRANSCRIPT
Protocol Analysis
• Knowledge provider works through some pre-defined task, KE observes.
• Varieties:– Think-aloud– Critical response– Periodic report– Report by commentary
Think-aloud protocol
• Knowledge provider performs task, “thinking aloud” as s/he does it
keeps elicitation on track, but:time consuming
knowledge provider may forget
knowledge provider unlikely to verbalise at appropriate grain level
Critical response
Knowledge provider gives description when sub-task completed
likely to generate a summarised set of necessary sub tasks
rule derivation may therefore be less time consuming
but:
summary information only
holes in the knowledge therefore possible
Periodic report
Knowledge provider reports, at given time intervals, what s/he is trying to achieve, and the steps leading to it
useful if nature of task precludes vocalising and working simultaneously
keeps provider’s concentration up
but time consuming
Report by commentary
Ask knowledge provider to watch video, and comment
General danger with protocol analysis - observer effect GOW
(“Hawthorne effect”)
Possible guidelines for PA sessions
• (Based on Schreiber et al 2000):– Present problems and data in a realistic way– Transcribe the protocols as soon as possible– Avoid long self-report sessions– Be present, even during think-aloud sessions
Multi-dimensional scaling
• Collect objects in the domain• Have expert manipulate these, and hence sort
into related groups according to different criteria(hence “multi-dimensional”) GOW
•• Hope thus to achieve understanding of the
domain ontology• Varieties:
– Card sort– Repertory grid
Card-sort
Essential idea:• obtain set of concepts that covers the requisite
domain• transfer each to a card• ask expert to sort each into groups, and identify
what each has in common• iteratively combine the groups to form a
hierarchy• shuffle cards and repeat
“Advantages” of card sort technique
• Quick and relatively easy way of establishing the global structure of a set domain area
• Easy to perform - KE has little involvement while cards being sorted
• Relevance assured• Can help uncover implicit knowledge
(Milton et al 1999)
“Disadvantages” of card sort technique
• Time/effort to prepare the cards• Danger of gaps in the hierarchical model -
assumes KE has got the cards right• (wise therefore to have some blank cards)• Debatable how much used:
– “virtually unused” (Wagner et al, 2003 p 82, but cf. ibid. p 80!)
– “invaluable part of the knowledge engineer’s toolkit” (Rugg and McGeorge, 2005 p 105).
– arguably useful re information system design, especially web sites (eg Robertson, 2001)
Repertory grids
Essential idea:• obtain set of concepts that covers the requisite domain• transfer each to a card• expert draws three at random• expert says which two are the most alike• says why (“identifies the construct” behind the decision)• repeat until reasonable number of constructs• expert then rates each object according to its position on
a bipolar scale for each of the constructs
“Advantages” of repertory grids
• can elicit similarities and subtle distinctions between concepts (Wagner et al 2003)
• computational tools available (Rugg and McGeorge 2005)
• extensively used (ibid.)
• can help uncover implicit knowledge (ibid.)
“Disadvantages” of repertory grids
• may require specialist statistical skill by the KE
• very demanding for provider, if many comparisons
• approach is of no use if data do not form semantic scales (ibid.)
Laddered grid
Aim: gain spatial representation of relationships between domain concepts (“domain ontology”), and information about definitions.
Approach:start at arbitrary point in the hierarchical concept space (the “seed” item)work up, down across the space, gaining information about superordinate and subordinate concepts
Hence 4 types of question likely to be asked:“What is <concept> an example of?” - probing for superordinate concepts“Can you give examples of <concept>?” - probing for subordinate concepts“Are there any other examples of <concept>?” - probing for concepts at same level
as named concept“What distinguishes <concept1> from <concept2>?” - probing for criterial attributes
Grid is drawn as the session proceeds
“Advantages” of laddered grids
• exploration of a domain can be quite rapid, and engineer obtains easily interpreted domain map
• may be able to move directly from ladder to rules and/or semantic net - no transcription
• little domain knowledge required by KE• can create hierarchies of knowledge elements
such as concepts, attributes, processes and requirements
• A “well established knowledge acquisition technique” (Yan et al 2005)
“Disadvantages” of laddered grids
• people differ in their ability to think “spatially”
• “a somewhat contrived technique” (Schreiber et al 2000)
• assumes domain of interest is hierarchically structured (Yan et al 2005)
Other approaches to knowledge acquisition
• Forward scenario simulation• Goal decomposition
– (similar - chapter listing)
• Inquisitive observation• Systematic symptom to fault links• 20 questions• Review • Machine learning
Summary
• Knowledge acquisition is difficult• But crucial for expert system development
– And arguably for “general” AI– Unless machine learning can come to the fore
• Various techniques for knowledge acquisition exist – and arguably are crucial re “knowledge management”– and also for requirements analysis (Rugg and
McGeorge 2005)• The techniques are by no means mutually
exclusive (ibid.)
More exam information
• There are 3 questions
• Question 1 is worth 40 marks (half of the exam) and was given out last week
• Question 2 contains 5 parts, each worth 4 marks
• Question 3 contains 3 parts (worth 6, 6 and 8).
Exam information – topics covered
• Introduction • Expert systems• Neural networks• Multi-Criteria Decision Modelling in Business• Knowledge Representation (I) – production rules• Knowledge Representation (II) – logic and frames• Uncertainty and risk• Decision Modelling: A Survey of Business Models• Knowledge Acquisition and Management I• Knowledge Acquisition and management II• KBDSS: Ethics, Business and Society I• KBDSS: Ethics, Business and Society II
Exam information – topics not in the exam
• Nothing on the final 2 weeks apart from question 1 (the one you already have)
• Nothing on uncertainty
• Nothing on MCDM