Case-Based Learning Case-Based Learning Mechanisms to Deliver Learning Mechanisms to Deliver Learning

MaterialsMaterials

Todd Blank, Leen-Kiat Soh, L. D. Miller, Suzette Todd Blank, Leen-Kiat Soh, L. D. Miller, Suzette PersonPerson

Department of Computer Science and Department of Computer Science and EngineeringEngineering

University of NebraskaUniversity of Nebraska

{tblank, lksoh, lmille, sperson}@cse.unl.edu{tblank, lksoh, lmille, sperson}@cse.unl.edu

The ILMDA Project: GoalsThe ILMDA Project: Goals

Build an agent capable of adapting Build an agent capable of adapting examples and problems to student examples and problems to student behaviorbehavior

Develop courseware for CS1 and CS2 Develop courseware for CS1 and CS2 computer science classescomputer science classes

Establish a flexible database for the Establish a flexible database for the ILMDA systemILMDA system• Intelligent Learning Materials Delivery Intelligent Learning Materials Delivery

AgentAgent

Learning MaterialLearning Material

TutorialTutorial• RecursionRecursion

ExamplesExamples• Towers of HanoiTowers of Hanoi

ProblemsProblems• What is the output of this method if we What is the output of this method if we

pass in 5 as an argument?pass in 5 as an argument?

Project FrameworkProject Framework

GUI front-endGUI front-end ILMDA reasoning moduleILMDA reasoning module Database backendDatabase backend

ILMDAReasoning

studentComputer& GUI

databaselectures

Historical profile,Real-time behavior Parametric profile of

student and environment

Retrieval instructionsProfile updatesStatistics updates

Timely deliveryof examples & exercise problems

ExamplesExercise problemsStatistics

ILMDA Agent

Project FrameworkProject Framework

Design FeaturesDesign Features

Utilization of true agent intelligenceUtilization of true agent intelligence Agent accountability of usefulness for Agent accountability of usefulness for

evaluationevaluation Modularization of course content and Modularization of course content and

deliverydelivery

Intelligence ModulesIntelligence Modules

Agent employsAgent employs• Case-based ReasoningCase-based Reasoning• Feedback from environment Feedback from environment • Meta-learns about good adaptation Meta-learns about good adaptation

heuristicsheuristics

Learning ModulesLearning Modules

Case Learning ModuleCase Learning Module Similarity Learning ModuleSimilarity Learning Module Adaptation Learning ModuleAdaptation Learning Module

Case Learning ModuleCase Learning Module

Finds most similar caseFinds most similar case• For successful cases, adapts on case with For successful cases, adapts on case with

Case-based reasoningCase-based reasoning• For non-successful cases, adapts on case with For non-successful cases, adapts on case with

Simulated AnnealingSimulated Annealing Checks success rate of each case stored in Checks success rate of each case stored in

databasedatabase• timesCaseUsed, timesCaseSuccessfultimesCaseUsed, timesCaseSuccessful• Unsuccessful cases have candidateForAnneal Unsuccessful cases have candidateForAnneal

set to trueset to true

Similarity Learning ModuleSimilarity Learning Module

Changes weights for computing Changes weights for computing similarity of two casessimilarity of two cases• Compares outcome for each case with Compares outcome for each case with

its most similar caseits most similar case• Weights are increased when cases have Weights are increased when cases have

similar outcomessimilar outcomes• Weights are decreased when cases have Weights are decreased when cases have

dissimilar outcomesdissimilar outcomes

Adaptation Learning ModuleAdaptation Learning Module

Changes weights for adaptation on Changes weights for adaptation on most similar casemost similar case• Considers outcome of previously used Considers outcome of previously used

casescases• For successful cases, slight changes to For successful cases, slight changes to

adaptation weightsadaptation weights• For non-successful cases, aggressive For non-successful cases, aggressive

changes to adaptation weightschanges to adaptation weights

Adaptation Learning ModuleAdaptation Learning Module

Checks success rate cases stored in Checks success rate cases stored in databasedatabase

Cases have been successfulCases have been successful• Slight changes to adaptation weightsSlight changes to adaptation weights

Cases have been unsuccessfulCases have been unsuccessful• More aggressive changes to adaptation More aggressive changes to adaptation

weightsweights

ImplementationImplementation

ILMDA agentILMDA agent• Java (Swing for GUI)Java (Swing for GUI)• PHPPHP• MySQL databaseMySQL database

Learning ModulesLearning Modules• JavaJava• MySQL databaseMySQL database

ImplementationImplementation

MySQL database housesMySQL database houses• Student InformationStudent Information• TutorialsTutorials• ExamplesExamples• ProblemsProblems• Agent HeuristicsAgent Heuristics• Performance StatisticsPerformance Statistics

Student Student Interaction Interaction with GUIwith GUI

A Case: Input FeaturesA Case: Input FeaturesInput Variables Description

AveExmpClicks The average number of times the student clicks the mouse in the examples he or she has seen. (When comparing two cases to pick a problem, the number of example clicks in that session is used instead).

AveExmpTime The average time spent (in milliseconds) per example. (When comparing two cases to pick a problem, the amount of time spent during the example clicks in that session is used instead).

AveExmpToTtrl The average number of times the student goes back to the tutorial from the example.

AveGrade The student’s average grade on the problems.

NumExmp, NumProb, TtrlClicks, TtrlTime, Self Efficacy, Motivation, etc.

..................

Parameters Description

DiffLevel The difficulty level of the problem or example.

MinUseTime The shortest anyone has looked at the problem or example.

MaxUseTime The longest anyone has looked at the problem or example.

AveUseTime The average time students view the problem or example.

AveClick The average # of times students click the mouse in the problem or example

Length The # of characters in the example or problem

Bloom The Bloom’s taxonomy value for the problem or example.

A Case: Output FeaturesA Case: Output Features

SimulatorSimulator

Virtual Student TypesVirtual Student Types• Student SpeedStudent Speed

SlowSlow AverageAverage FastFast

• Student AptitudeStudent Aptitude KnowledgeableKnowledgeable AverageAverage Not KnowledgeableNot Knowledgeable

ExperimentsExperiments

ILMDA interacts with Virtual studentILMDA interacts with Virtual student Measure average quitting point and Measure average quitting point and

outcome for Virtual studentsoutcome for Virtual students• Quit at tutorial (0), Quit at example (1), Quit at tutorial (0), Quit at example (1),

Quit at problem (2), Quit after problem Quit at problem (2), Quit after problem (3), Answer problem correctly (4)(3), Answer problem correctly (4)

Measure correct answer scoresMeasure correct answer scores

ResultsResults

1000 simulations with and without 1000 simulations with and without learning moduleslearning modules• Without learning modulesWithout learning modules

1.827 average quitting point, .056 average score1.827 average quitting point, .056 average score

• With learning modulesWith learning modules 1.882 average quitting point, .116 average score1.882 average quitting point, .116 average score

Observation (Marginal)Observation (Marginal)• ILMDA was able to give better examplesILMDA was able to give better examples• ILMDA was able to give problems that virtual ILMDA was able to give problems that virtual

students answered correctly more oftenstudents answered correctly more often

Ongoing WorkOngoing Work

Deployed ILMDA in CSCE155, Fall 04Deployed ILMDA in CSCE155, Fall 04• Five topics: Simple Class, Exception, Event-Five topics: Simple Class, Exception, Event-

Driven Programming, Driven Programming, Inheritance/Polymorphism, RecursionInheritance/Polymorphism, Recursion

• Agents with and without learning mechanismsAgents with and without learning mechanisms To find the impact of learningTo find the impact of learning To identify features To identify features To identify key adaptation heuristicsTo identify key adaptation heuristics To identify useful casesTo identify useful cases

• Results collected but yet to be analyzedResults collected but yet to be analyzed