tutoring and help systems tell me and i forget. show me and i remember. involve me and i understand....
TRANSCRIPT
Tutoring and Help Systems
Tell me and I forget.
Show me and I remember.
Involve me and I understand.
- Chinese proverb
Previous Approach
• Used for over 20 years– Computer-based training (CBT)– Computer aided instruction (CAI)
• Effective in helping learners, but do not provide the same attention a human tutor can provide– Approach to a solution– Individual problem solving style
A New Approach
• Previous approaches focused on scripted information about the domain
• New approach must reason about both the domain and the learner
• Allowing greater versatility in systems interaction with students
Intelligent Tutoring System
Possesses two intelligent properties:
1. Generate problem solution– Flexibility within problem domain– Able to explain errors
2. Adapt to user needs– User knowledge models
Model of Traditional ITS
Components in Tutoring System
Domain Knowledge
Expert Model
Pedagogical Module Communication
Model
Student Model
Student Model• Used to tailor instruction for each student• Must represent the student’s knowledge with
respect to domain– Choice of representation
• Must store pedagogical information about the student– Student’s preferences– Problem solving style
• General information– Acquisition and retention
Student Model Representation
• Typically represented with overlays
• Student’s knowledge as a subset of expert’s knowledge
Overlay Student Model
Student’s
Knowledge
Expert’s Knowledge
Overlay With Buggy Extensions
Expert’s Knowledge
Student’s Buggy
Knowledge
Shared Knowledge
Pedagogical Module
• Uses information from the student model to determine what to present to learner– New Material from the Domain– Review of Previous Topic– Feedback on Current Topic
• Teaching Meta-Strategy vs. Low Level Issues
Low Level
• Topic Selection– Examine student model for areas of focus
• Problem Generation– Difficulty based on student’s ability level (taken from
student model)
– Size of question depend on granularity of domain
• Feedback– How much & What kind
Meta-Strategy• Implementing strategy has been a
formidable problem• Ideal to have many strategies to choose
from based on student model– Realistically many ITSs only have one
• Difficulty in representing knowledge impedes some methods– Socratic method requires “Common Sense”
…Enter CBR
• Individualizing will depend on two issues1. Information about how learner solved tasks
2. Using this information in subsequent tutorial decisions
• Storing this information builds cases• Cases from other learners• Pre-stored cases - Pitfalls domain experts
have foreseen
Two Goals of CBR Tutoring
• Case-based Adaptation– Adapt interface components to the user’s needs
– CBR that not only uses pre-stored cases but also stores new cases can be adapted
– CHEF: Recipe and Taste
• Case-based Teaching– Provide user with cases that help solve current problem
– Observe user solving problem – cases can be used as a reminder
Two Ways to Store a Case
1. Case is stored as a whole– Most systems use this approach– Show examples or give advice
2. Case is stored as a snippet– Describes sub goals of problems within
particular context– Used to find problem solving path– Application used in ELM
Episodic Learner ModelELM
• Analyzes solutions (or partial solutions) to programming problems in LISP
• Looks for problem solving errors and returns feedback
• Used in diagnostic process
• Able to return examples and remindings– EBR: Explanation-based retrieval
ELM
• Stores user model in a collection of episodes (cases)
• User code is analyzed to create a derivation tree consisting of concepts and rules
• These concepts and rules are instantiations of units from the knowledge base
ELM Knowledge Representation
• Represented in hierarchically organized frames• Concepts
– Knowledge about the language (concrete procedures and semantic concepts)
– Schemata of common algorithmic and problem solving knowledge (eg recursion)
• Additional information– Plan transformations for semantically equivalent
solutions• Bug rules for derivations which may result from
confusion
Bug Rule•Bug Code
•Ideal Code
Append:(APPEND “a” “bcd”)(APPEND (a) (bcd))
Append:(APPEND ‘(a) ‘(bcd))
ELM Diagnostic
• Code is at least syntactically correct• Starts with task description related to higher
concepts in the knowledge base• Most concepts have transformations describing
semantically equivalent variations– Ordering of clauses or sequence of arguments
• The sequence of testing transformations is determined by the student model
ELM Diagnostic Cont.
• A set of rules is indexed by concepts describing different ways to solve the goal– Good– Bad– Buggy
• Applying a rule results in comparison between plan and student’s code
• Diagnostic process is called recursively on further concepts– Results in derivation tree
ELM Derivation Tree
• Information in tree added to episodic model– Instances of concepts and rules
• Context
• Transformations and argument bindings
• Each concept (level) in tree creates a frame• The set of episodic frames of a particular
episode constitutes a case– Can later be indexed by first frame in case to
rebuild tree
Partial Derivation Tree:
(NIL-TEST(FIRST-ELEMENT(PARAMETER?LIST)))
NIL-TEST
Empty-List-Nil-Test-Rule
(NULLTEST(FIRST-ELEMENT(PARAMETER?LIST)))
NULLTEST
Unary-Func-Rule
(NULL-OP) (FIRST-ELEMENT(PARAMETER?LIST))
NULL-OP FIRST-ELEMENT
Correct-Coding-Rule Unary-Func-Rule
null (FIRST-ELEM-OP) (PARAMETER ?LIST)
FIRST-ELEM-OP PARAMTER
Correct-Coding-Rule Correct-Param-Rule
car li
Student CodeSimple And:(defun simple-and(li) (cond ((null li) t) ((null (car li)) nil) (t (simple-and (cdr li)))))
Derivation Tree
ELM
LISP Code
Diagnosis(Explanation)
Derivation Tree(Explanation Structure)
Task Description
Domain Knowledge
Learner Model Generalization
Explanation-Based Retrieval
• System generates a solution based on concepts and rules and temporarily stores this solution in case base
• All episodic frames that are neighbors contribute to computing weights for similarity
• Most similar case is retrieved (based on previous explanations) and temporary solution is deleted
ELM-Programming Environment
• Intelligent analysis of task solutions– Diagnostic tool based on ELM– Gives user feedback on purposed solution– Directs user with hints
• Example-based Programming– Can reuse code from pre-installed cases or the user’s
own previous experience
• Example-based Explanation– Shows examples based on matching of expected
solution with previous cases already in learner model
ELM-Adaptive Remote Tutor
• HTML Implementation of ELM-PE• Conceptual network of topics
– Red light, green light
• Example-based programming– Can find the most relevant example from case history
• Demonstration of ELM-ART– http://apsymac33.uni-trier.de:8080/elm-art/login-e
Static vs Dynamic CB Teaching• Static
– Problem design facilitates the diagnosis of failure– Cases (failures) are associated with supporting case to
help overcome failure– Limited by case-base
• Dynamic– Problems solved twice, by learner and system– System solution used as an index for supporting cases
• Model Tracing– Similar to dynamic, but solution used for direct
feedback (could limit multiple solution paths)
Case-based Chess Endgame Tutor
• Dynamic Teaching• Chess heuristics are not given, instead must be
inferred– First given examples to watch
– Next examples to solve
• CACHET structures this learning by recognizing sub optimal moves and providing hints that lead in the right direction
CACHET Case Libraries
• Pre-defined cases– Prototypical informative games
• Cases generated on demand– Able to generate scenarios for learner
• Cases produced by the learners themselves– Self-generated cases are very successful for
remindings– Useful to system as point of intervention
Roger Schank
• President and CEO of Socratic Arts
• Founder of Institute for the Learning Sciences
• Research on AI and cognitive learning theory
• Focus on e-learning
Cognitive Learning Theory
• A general approach that views learning as an active mental process of acquiring, remembering, and using knowledge.
• Learning is evident by a change in knowledge which makes a change in behavior possible.
• Learning itself is not directly observable.
Schank’s Criticisms• Schools act as if learning can be disassociated
from doing• Schools believe they have the job of assessment
as part of their natural role• Schools believe they have an obligation to create
standard curricula• Schools believe studying is an important part of
learning • Schools believe students have a basic interest in
learning whatever it is schools decide to teach them
Schank’s Idea
• Case-based reasoning: Understand the universe by matching incoming events to past experiences– The Steak and the Haircut
• Knowledge is built on the ability to index and make sense of cases– It is not a set of facts!
• You must question to learn
Conclusion
• CBR can effectively be applied to enhance tutoring systems
• Cases can be complete or snippets• Cases can include buggy information• Cases are applied either diagnostically or
adaptively• ELM-PE and ELM-ART use cases
diagnostically forming a derivation tree