Knowledge Acquisition and Modelling
Knowledge Acquisition and Elicitation
Ref: Knowledge Acquisition in Practice: A step by step guide, Milton, Springer-Verlag
Knowledge Engineering Transfer View
Human knowledge transferred to knowledge base =>knowledge exists and is accessible Typically interviews and task execution and
observation used for KA End result set of rules that exercise knowledge
made explicit Modelling View
Need to build models Incremental, evolutionary process Model is an approximation of reality Models are subjective
KA Typology
KA techniques
natural techniques
contrived techniques
modelling techniques
interviews
observation techniques
group meetings
questionnaires
unstructured interview
semi-structured interview
structured interview
card sorting
three card trick
rep grid technique
constrained tasks
20-questions
commentating
teach back
limited time
limited information
laddering
process mapping
concept mapping
state diagram mapping
Natural Techniques
Interview Techniques Knowledge engineer asks questions of the expert or end
user. Essential method for acquiring explicit conceptualisations
and knowledge, but poor for tacit knowledge. Variations:
Unstructured interview Free flowing, used in early stages of elicitation, can produce basics of
knowledge domain, basically broad chat Semi-structured interview
Main technique for elicitation Pre-defined questions sent to expert prior to interview,
supplementary questions asked at interview. Can be used as part of validation.
Structured interview Pre-defined set of questions, can simply be filling in a questionnaire
at the interview.
Interview Techniques Dependent on
The questions asked Ability of the expert to articulate the knowledge
Model built on knowledge elicited during interview
Model reviewed by the expert
Modelling Techniques
Modelling Techniques Use of knowledge models with experts Used as validation and refinement Can show a basic model to an expert and
prompt them to modify. Can show a complete model of knowledge
provided by one expert to a second expert to cross-validate.
Can create one from scratch with an expert – start with a blank page
Model Based Knowledge Acquisition Each model emphasizes certain aspects of the
system to be built and abstracts from others. Each model is indicative of one view of the
world Models provide a decomposition of
knowledge-engineering tasks: while building one model, the knowledge engineer
can temporarily neglect certain other aspects.
Knowledge Modelling Process
Knowledge Modelling Use skeletal models Or generic tasks
Generic tasks are templates of problem-solving activities that can be configured together to describe any intelligent activity.
Modelling Frameworks
Knowledge Modelling At least five different types of knowledge are
distinguished: Tasks-goals
correspond to the goals that must be achieved during problem solving. Problem-solving methods
ways to achieve the goals described in tasks. In some knowledge modelling frameworks, problem-solving methods define subtasks to which other problem solving methods can be applied. We will call such a decomposition a task instance.
Inferences describe the primitive reasoning steps in the problem solving process.
Ontologies describe the structure and vocabulary of the static domain knowledge.
Domain knowledge refers to a collection of statements about the domain.
Principles Divide and conquer. Configuration of an adequate model set for a
specific application. Models evolve through well defined states. The model set supports project management. Model development is driven by project
objectives and risk. Models can be developed in parallel.
Recommended ReadingKnowledge Engineering: Principles and MethodsRudi Studer, V. Richard Benjamins and Dieter
Fense Data & Knowledge Engineering (1998)Volume: 25, Issue: 1-2, Publisher: Elsevier http://www.hubscher.org/roland/courses/hf760
/readings/studer98knowledge.pdf
Contrived Techniques
Knowledge Capture – Specialised Techniques Contrived Techniques Primarily for deep, tacit knowledge Involve the expert performing tasks they
would not normally do as part of their job. Most of these techniques come from
psychology
Knowledge Capture – Specialised Techniques Important types:
Concept (card) Sorting Three Card Trick (Triadic) Repertory Grid Technique Constrained Tasks 20-questions Commentary Teach Back
Usually involve expert doing two types of task: Tasks they normally perform
Commentary is useful here Tasks designed to probe the expert
Concept sorting or Triadic
Concept (Card) Sorting Way of finding out how an expert compares and orders
concepts Can reveal knowledge about classes, properties and relations
Works best in small groups Simplest form is card sorting
Collection of concepts (or other knowledge objects) are written on separate cards
Cards sorted into piles by an expert in to piles - each card in a pile must have something in common
Each time the cards are sorted it will be based on an attribute and each pile will represent a value
Enables significant elicitation of properties and dimensions Used to capture concept knowledge and tacit knowledge Use in conjunction with triadic method Can also sort objects or pictures instead of cards
Concept Sorting – How To ? Decide what classes of concepts you want to
explore (in particular their properties – attributes and values)
Write the name of each concept on a separate card
At the session explain to the expert what is going to happen
Ask the expert to name the piles Write down (or record) the results of the sort Collect the cards and ask the expert to sort
again Repeat until the expert can’t sort anymore
Triadic Elicitation Method (3 card trick) Used to capture the way in which an expert views the
concepts in a domain. Present three random concepts and ask in what way two
of them are similar but different from the other one. Answer will give an attribute. A good way of acquiring tacit knowledge. How does it work ?
Select 3 cards at random Identify which 2 cards are the most similar?
– Why? – What makes them different from the third card?
Helps to determine the characteristics of our classes Picking 3 cards forces us into identifying differences between
them There will always be two that are “closer” together Although which two cards that is may differ depending on your
perspective
Triadic Elicitation – How To? Explain to the expert that you are trying a technique to
draw out deeper knowledge Collect all cards previously used Shuffle cards and randomly select 3 Place them on the table, two close together one further
away Ask how the two close together are similar and the other
different Write down (or record) what the expert says using an
attribute Use the results to find another card sort to find the values
of all concepts for this attribute If the expert can’t identify an attribute, just pick another
3 cards Repeat until the expert can think of no more differences
20-Questions Expert asks questions of the engineer The Knowledge Engineer thinks of an object/concept in the
domain Expert asks yes/no questions to the knowledge engineer in
order to deduce an answer. Knowledge Engineer
notes the questions and the order in which they are asked need not know much about the domain, or have an answer in mind,
just answer “yes” or “no” randomly The questions asked provide a good way of quickly acquiring
attributes in a prioritised order. Can provide an insight into the key aspects, properties or
categories and their relative priorities. Note that the main purpose of this exercise is not really to try
and find out what the Engineer is thinking of, but to determine the important properties!
20-Questions – How To? Decide on set of concepts you need to explore in more detail Explain to the expert what is going on Ask the expert to imagine that you the engineer have the same level of
knowledge they do about the set of concepts Instruct the expert that they should ask the least number of questions
to deduce the answer Engineer can only answer yes and no Explain that the best way is to ask questions which split the concepts
in half so that each question eliminates half the possible solutions Start As each question is asked write it down (or record it) When a number of questions have been asked take the expert back to
an earlier question and change the answer you gave to prompt the expert to ask further questions
After the session extract the attributes and values (or new concepts) from the questions asked and these will be added to the knowledge base
Laddering Involves the construction, modification and validation of
trees. Accessing personal construct system Take a group of things and ask what they have in
common Then what other ‘siblings’ (brothers/sisters) there might
be A valuable method for acquiring concept knowledge and,
to a lesser extent, process knowledge. Can make use of various trees:
concept tree composition tree attribute tree process tree decision tree cause tree
Example
Source: Bourne and Jenkins , Eliciting Managers' Personal Values: An Adaptation of the Laddering Interview Method, Organizational Research Methods, SAGE 2005
Concept Tree Hierarchical diagram of concepts showing classes and
members Activities to create
Move nodes (concepts) around the tree Add new node Deleting nodes Renaming nodes
Difficulty is avoiding the problems which requires knowing: All links on the tree represent an ‘is-a’ relationship Terminology to describe the tree What classes to use in the tree Naming conventions to use How to deal with complex cases (e.g. multiple parents,
synonyms)
Concept Tree – ‘is-a’ relationship
Is-a = is a type of Different to ERDs
vehicle
traffic
ship
traffic issues
lorry
car
steam ship
sailing ship
shipping lanes
pollution
congestion
Road safety
What are the
mistakes in this tree?
Concept Tree - Terminology Root node Leaf node Branch Parent Children Descendants
Concept Tree – What classes to use? Class is a concept which has children on a tree Other concepts are related to it by an is-a
relationship To develop classes use either a top-down or
bottom-up approach Top-down start with a set of general classes and
refine Bottom-up start to develop classes by grouping
those concepts that are similar
Repertory Grid technique Used to elicit attributes for a set of concepts Used to rate concepts against attributes using a
numerical scale Uses statistical analysis to arrange and group
similar concepts and attributes Allows the expert to provide a rating of each
concept for an attribute in concept sorting A useful way of capturing concept knowledge and
tacit knowledge When many ratings are provided using many
attributes statistics can be applied to find clusters and correlations
Requires special software
Repertory Grid – How To? 1st stage
Concepts are selected (between 6 & 15) Set of approx. same no. of attributes is also required
Should be such that values can be rated on a continuous scale (e.g. small to large)
Chosen from knowledge previously elicited
2nd stage Expert provides a rating for each concept against each attribute Numerical scale is used
3rd stage Ratings are applied to cluster analysis to create a visual representation of
the ratings called a focus grid Concepts with similar scores will be grouped together, attributes with
similar scores will be grouped 4th stage
Engineer walks expert through the results to gain feedback and prompt for further knowledge about the groupings
If needed more concepts and attributes are rated and included in the grid
Repertory Grid Example
Domain elements are certain types of crime: petty theft, burglary, drug-dealing, murder, mugging and rape.
This is one expert’s view on the issue. Consider carefully whether any pair of
attributes are very similar, by comparing horizontal lines in this grid. The closest is probably the personal/impersonal
one and the major/petty one. Beware, when making this comparison, that
the expert may have inadvertently ‘inverted’ the scale for just one of two similar constructs. For example, in the example the major/petty
construct has a value of 5 for ‘major’. If the expert had chosen 1 instead, and 5 for ‘petty’, then this construct and the personal/impersonal one would look very different.
Further analysis may lead you to omit one pairing of constructs.
Following that you would draw up a table showing how similar or dissimilar each domain element is from the others.
For example, when the absolute-value metric is used, the (numeric) difference.
Constrained Tasks Expert performs a task they would normally
do, but with constraints. Variations:
limited time limited data
Useful for focusing the expert on essential knowledge and priorities
Commentary and protocol generation Expert provides a running commentary of
their own or another’s task performance. A valuable method for acquiring process
knowledge and tacit knowledge. Variations:
self-reporting imaginary self-reporting self-retrospective shadowing retrospective shadowing
Knowledge Analysis and Modelling
Knowledge Analysis Identifying the elements needed to build the
knowledge base Concepts
Things that constitute a domain Main elements of the k-base
Attributes Qualities or features belonging to a class of concepts
Values Specific qualities or features of a concept that
differentiate it from other concepts Relations
Way in which concepts are associated with one another
Concepts Physical concepts
Products, components, machines
Pieces of information Plans, goals, requirements
Sources of information Documents, databases,
websites People and roles
Experts, roles of experts Organisations and groups
Producers, suppliers, consumers, departments
Areas of knowledge Marketing, physics,
chemistry
Functions Purpose of components or
roles Tasks
Activities performed by experts
Issues Problems, solutions,
advantages, disadvantages Physical phenomena
Mechanisms and forces Other issues
Constraints, behaviours, states
Attributes Of physical objects
Shape, age Of information
Source, format, importance Of people
Gender, age, personality Of organisation
Size, turnover, product range
Values Come in different varieties Dependent on type
Adjective, number, sentence, paragraphs, hyperlinks, images, pictures
Categorical For values that are adjectives
Numerical For values that are numbers
Text For values that are one or two sentences
Hypertext For values that are chunks of hypertext
Relations Has part Performs Followed by Requires Causes Produces Can have an inverse relation Short exercise
Think of something that illustrates each one of these
Knowledge modelling K-model = way of viewing the knowledge in
the k-base Each model provides a different perspective
on the knowledge Helps clarify the ‘mess’ that is the knowledge Can be used in elicitation
Trees Diagram showing hierarchical arrangement of
nodes Node = concept Link = relationship Concept tree Composition tree Cause tree Mixed tree
Concept tree Each link is an is-a
relation Taxonomy Read from right to
left
Taken from www.pcpack.co.uk
Other types of tree Composition tree
All links are has-part Used to show components and sub-components of a
concept Process tree
Special form of composition tree All nodes are tasks
Attribute tree Shows attributes and values to describe a concept
Mixed tree Contains more than one type of relation
Matrices
Attribute matrix Presents set of
properties of a concept (attributes and values)
Concepts on vertical axis
Attributes and values on horizontal axis
Relationship matrix Shows two sets of
concepts related to one another using a specified relationship
Cells show which pairs of concepts have the relationship
Maps Shows an arrangement of nodes linked by
arrows Each node represents concept Link represents relationship Concept maps Process maps
Concept map
Many different types
Knowledge Analysis – How to? How do you identify concepts from interview
transcripts and documents? Need some codification Highlighters – different colours for different
things
Typical project 47 steps proposed by Milton Knowledge Acquisition in Practice: A step by
step guide, Milton, Springer-Verlag Phase I
Start, scope and plan the project Phase II
Initial capture and modelling Phase III
Detailed capture and modelling Phase IV
Share and store knowledge
Phase I – Start, Scope and Plan Identify a project
How it can benefit, what it involves Gather opinions from relevant people Document ideas as project proposal Seek agreement on proposal from key people Start knowledge capture With domain experts break the domain into different topics
and rank against key criteria Identify a proposed scope and finalise Identify sources of knowledge Define and understand the type of project to be able to
create a schedule Collate the proposal, scope and schedule into a project
plan and disseminate with other materials to team
Phase II – Initial Capture and Modelling Learn the basics of the domain from documents and
informal conversation with experts Prepare for semi-structured interviews then execute and
transcribe Analyse results to identify concepts, create a concept
tree to develop a taxonomy and validate with experts Create a k-page for each concept
K-page = 2 column table showing all knowledge associated with a concept
Create a glossary Build a meta-model showing the relationships between
concepts and relationships Build appropriate k-models Continue with validation models
Phase III – Detailed Capture and Modelling Use further interviews and modelling to
capture more detailed knowledge Finalise k-model Prepare prototype end product used to carry
out assessment exercise with sample of end-users
Produce a completion plan defining what needs to be done to complete the project
Use specialised techniques to do detailed knowledge capture
If needed cross-validate between experts and resolve conflicts
Phase IV – Share and store knowledge Define and create format of end-product Create provisional end-product using k-base Give to experts for full validation Create final end-product and release for use After use for some time assess impact on
organisation and document it Conduct complete product review to learn
lessons and make suggestions to change methodology
Ensure end-product is useful, usable and used End-users must find
Find product useful Find product easy to use Actually use it