July 6, 2004 7th International Protégé Conference
Reasoning in a Tutoring System: Transforming
Knowledge to Teaching.
Olga Medvedeva
Center for Pathology Informatics,University of Pittsburgh
July 6, 2004 7th International Protégé Conference
Outline
• Our approach for teaching visual diagnosis
• General system architecture
• Knowledge representation in different tutor modules
• Pluses and minuses of our system
July 6, 2004 7th International Protégé Conference
Medical KB Training System Challenge
• Problems – Medical knowledge is complex and dynamic– Errors in KB can cause serious problems– Demands on extendibility and maintenance of
large KBS
• Requirements– Combine knowledge representation and flexible
instructional system– Adaptive for new observables and unique
strategies– Reusable and modular
July 6, 2004 7th International Protégé Conference
Intelligent Tutoring SystemsParadigm
ITS strive to replicate a method of teaching and learning exemplified by a one-on-one human tutoring interactionModel Tracing ITS guide user through
problem space, can correct each small intermediate reasoning step Cognitive Tutors based on ACT_R
theory of learning proceduralize declarative knowledge in the rules(step instructions)
July 6, 2004 7th International Protégé Conference
Student Module
Pedagogic Knowledge
Interface
Expert Model
•Allow correct steps•Correct errors•Give hints on next step
•Collect data on what student does•Make predictions on what student knows•Provide data for pedagogic decision making
•Canvas for problem solving•Make goals visible
•Case sequence•When to intervene•How to intervene
Intelligent Tutor System Structure
July 6, 2004 7th International Protégé Conference
Disadvantages of ITS Paradigm
• Developed for highly procedural domains• Not designed for large complex dynamic
declarative knowledge• Domain specific production rules
knowledge representation• Maintenance is difficult and time
consuming• Knowledge modification alter the rules
July 6, 2004 7th International Protégé Conference
SlideTutor Characteristics
SlideTutor – a system to teach visual classification problem solving in Pathology
• Similar to other medical diagnostic tasks– Combination of search, identification, interpretation– Well characterized diagnostic reasoning in medical
domains– Some areas are highly algorithmic, some – not– Both empirical and theoretical work can guide the
development– Combination of heuristic classification and
deductive/inductive reasoning is the best foundation for classification problem-solving.
July 6, 2004 7th International Protégé Conference
SlideTutor Approach
• Combine the aspects and methodology of both KBS and ITS to create a general framework for teaching decision-making process for classification problems in Dermopathology using UPML Component Mode approach.– Extract and modularize all expert and pedagogic
declarative knowledge into ontologies => make domain task neutral
– Reuse PSM by tutor procedural rule based system => make system domain neutral
– Preserve all of the major pedagogic components associated with Cognitive Tutors in ontologies and rules => add significant flexibility to pedagogic model
July 6, 2004 7th International Protégé Conference
Instructional Layer
Pedagogic TaskStructure
PedagogicTask
DermatologyKnowledge Base
DomainModel
VisualClassification
Task Structure
DomainTask
PedagogicKnowledge Base
PedagogicModel
Case Database
Interface
Expert Model
Student Model
StudentModelState
StudentModelStateStudent Model
Data
SlideRepresentation
Case Data
SlideRepresentation
Case Data
SlideRepresentation
Case Data
Student
Dynamic Solution Graph
Pedagogic Model
DomainBehaviorRefiner
ProblemSolvingMethods
PedagogicBehaviorRefiner
ProblemSolvingMethods
SlideTutor General Architecture
July 6, 2004 7th International Protégé Conference
Domain Model
• Set of ontologies that express relationships between evidence and disease concepts
• Uses Motta’s parametric design approach (slightly extended by adding attributes to features)
• Similar disease and evidence representation– Hierarchical structure with multiple inheritance
for diseases– Set of evidences represent set of diseases– Both can occur multiple times in different sets
July 6, 2004 7th International Protégé Conference
Feature – Domain KB – Case Relationship
July 6, 2004 7th International Protégé Conference
Task Model
Models the abstract structure of the Dynamic Solution Graph (DSG) – a directed acyclic graph
• Represents possible relationships in the domain knowledge that are pertinent for reasoning– Identifying region– Identifying and refining a set of features– Triggering one or more hypothesis– Creating a differential diagnosis– Finding features that distinguish between the hypotheses– Defining that critical feature is absent– Linking supportive features to a particular hypothesis– Accepting some hypotheses as diagnosis
• Direction of DSG is defined by an order of some steps in task
(deftemplate task
(slot type)
(multislot parent)
(slot role)
(slot required)
(slot priority))
July 6, 2004 7th International Protégé Conference
JessTab Extensions
• Added UserFunctionsload-jdbc-project - load db projectdisposep - dispose current Protégé
• Modified code– Preserve class hierarchy structure– Multiple inheritance (MAIN::NEUTROPHILS
(is-a NEUTROPHILS) (is-a-name "NEUTROPHILS") (OBJECT <External-
Address:edu.stanford.smi.protege.model.DefaultSimpleInstance>) (has-parents "INFLAMMATORY INFILTRATE")
(feature_name "isolated neutrophils"))
July 6, 2004 7th International Protégé Conference
Dynamic Solution Graph
• Generates path through problem state based on combination of Domain, Task and Case models
• Dynamic – no predefined solution – each cycle generates the current problem state and all valid next steps
• Contains a set of abstract PSM that allow to add/delete/update nodes and arcs
• Path through the problem is defined by a consequence of student actions
• Behavior structures encapsulate node type specific response to a triggered event
• Supports forwards and backwards reasoning
July 6, 2004 7th International Protégé Conference
DSG Implementation
(deftemplate node
(slot type (type STRING))
(multislot property_name ) ;; e.g “name” “x” “y” “z”
(multislot property_value)
(slot internal_id (type STRING))
(slot state (default " INITIAL ")) ;; INITIAL, IDENTIFIED
(slot input_value) ;; easy match with useraction input slot
(slot external_id (default nil)) ;; id of a corresponding object on user side
(slot is_goal (default FALSE))
(slot is_from_case (default FALSE))) ;; node can not delete if came from case
• Node reflects the semantic meaning of fact• Correct student action must match all of the node properties• State indicates that step was performed by user or not• Interpretation of action is left to the instructional layer• Special node type – Cluster node – expresses integrated
relation between a specific group if nodes and nodes outside it
July 6, 2004 7th International Protégé Conference
July 6, 2004 7th International Protégé Conference
DSG Cognitive Values
• Enables rapid feedback • Provides a method for stepping forward in
the model to generate next-step hints• Supports intermediate solution and revision• Determines general classes of errors and
allows pedagogic model to remediate them• Provides flexibility in tutor response• Reusable, because domain and
pedagogically independent
July 6, 2004 7th International Protégé Conference
Instructional Layer
• Pedagogic Model– Explanation of a particular student error and rich
next-step hints upon student request– Delivered messages contain context-specific text
accompanied by the pointers to the places of interest on the user side
– Determines the most appropriate error from the error list generated by the DSG as a response to incorrect student action based on the state of student model
– Hierarchical hints from general to most specific and directive
• Pedagogic Task – represents the goal of the instructional process
July 6, 2004 7th International Protégé Conference
Case-Focused Interface
• Local view of the problem
July 6, 2004 7th International Protégé Conference
Knowledge-Focused Interface
• Global view of the problem (use SpaceTree cs.umd.edu)
July 6, 2004 7th International Protégé Conference
Conclusion
• Preserved essential characteristics of CT • Utilized KB for modeling knowledge across the
system components• Modular and flexible set of frames and
methods to teach classification problem solving
• Limitation – deterministic approach– No support for probabilistic relationship between
evidence and hypothesis– No attempt to model all evidence combinations or
incomplete evidence– No reasoning under uncertainty
July 6, 2004 7th International Protégé Conference
Acknowledgements
• NLM 1 R01 LM007891-01 (Crowley, PI)
• Rebecca Crowley, Pathology Informatics• Eugene Tseytlin, Pathology Informatics • Elizabeth Legowski, Pathology Informatics• Gerish Chavan , Pathology Informatics• Maria Bond , Pathology Informatics
July 6, 2004 7th International Protégé Conference
More details at Demo Session
• Integrating Protégé into an Intelligent Medical Training System– Ontologies– Knowledgebase Validation Tool– Case Authoring Protégé plug-in– Dynamic Solution Graph– Protocol Filter Query– SlideTutor– DinoTutor