Math Fundamentals Tutor to Improve Training

Beverly Park Woolf, Ivon ArroyoDepartment of Computer Science

University of MassachusettsBev@cs.umass.edu

Wayang Tutoring Systemhttp://Wayangoutpost.com

3

We use an adaptive intelligent instructional system

4

Educational Goals

Provide each student with alternative• representations of content• paths through material • means of interaction

Move away from ‘one-size-fits-all’ education, away from passive lecture style teaching

Match the needs of individual students

Support active learning

FREE Online System: 300 problems

Wayang Tutoring Systemhttp://Wayangoutpost.com

Most Students achieve Higher Results on State Standard Exams.

MCASpassing%

WITH Wayang

MCASPassing%

NOWayang

77% 60% **

34% 24% *

92% 76% *

WayangPosttest

ControlNo Wayang

76% 67% **

d=0.25

d=0.24

d=0.52

Empirical Learning Results Since 2003

After short exposure (3-4 hours)

College-LevelClasses

3. Emotion is central to learning

Neuroscience teaches us that the emotional brain (Limbic system) has the power to open or close access to learning, memory and the ability to make novel connections. Excitement/arousal is required for learning.

Negative emotion drains intellectual energies. Scared children perform poorly; students faced with frustration, despair, worry, sadness or shame, loose their ability to learn.

Computational tutors can measure student emotion, based on sensors.

Why is emotion important to

learning?Emotions/Attitudes/Affect are important long-term outcomes.

.In general, students are really bored

with mathematics.

Emotions/Attitudes/Affect are important long-term outcomes.

Also, there are important group differences in students

emotions, before tutoring.

11

Detecting Student Emotion Using Sensors

Student behavior?

Excitement

Frustration

Boredom

Correct, but little effort

Quick-guess incorrect

Effort and Time on Hints

Tutors recognize student emotion

Tutors respond by changing the problem or providing supportive help.

Affective learning companions congratulate students on effort exerted and talk to them about their effort and learning.

Affective Learning Companions

Incorrect ResponseStudent effort shown/correct response

Student effort shown /incorrect response

Agent Emotion

Agents support frustrated students by acting helpful, bored, or confused.

Arroyo et al., AIED2009

Agent EmotionEffort Attribution Shrug High interest

Students believe agents are part of the learning experience, mentors. . . who are together with students against the computer, . . . who are more knowledgeable (most of the time) cognitively and emotionally.

Arroyo et al., AIED2009

Methodology

Measure students’ cognitive and affective attributes, (skills, motivation, engagement) in real-time.

Offer appropriate and timely interventions.

Measure the impact of each intervention

Frustrated Pretest Frustrated Within Tutor

Frustrated Posttest1.5

2

2.5

3

3.5

4

4.5

5

No Learning CompanionLearning Companion

How

FRUS

TRAT

ED d

o yo

u fe

el w

hen

solv

ing

mat

h pr

oble

ms?

Reduced Frustration

More Frustrated

Less Frustrated

NeutralFrustration

Level

Increased InterestLess boredom for math at posttest time in LC condition.

Interested Pretest Interested Within Tutor Interested Posttest0.6

1.1

1.6

2.1

2.6

3.1

3.6

4.1

No Learning Companion Learning Companion

How

INTE

REST

ED a

re y

ou w

hen

solv

ing

mat

h pr

oble

ms?

+F(94,1)=3.4,p=.07

More Interested

More Bored

NeutralInterest

Level

Improved Confidence

A computational system can trace students at the affective level and understand their emotions using indicators from recent behaviors, and physiological sensors.

Summary: Emotion is Critical

4. Individualized Instruction Supports Learning

Neuroscience teaches us that an instructional event must be salient to be remembered and students must be engaged to learn.

Emotion helps learning. Neurons must fire to be wired. Synaptic plasticity (growing new synapses) suggests that neurons can change, providing hope that every person can learn.

Computational tutors provide tight cycles of learning, assessment, replanning and rehearsal by providing instant feedback to teachers, classified by individual student, skill level, topic and problem.

Zone of Proximal Development”

Lev Vygotsky: the foundation of cognitive development, particularly of Social Development Theory. Adaptive learning means a

computational system maintains a student within their zone.

Murray, T.; Arroyo, I. (2002) Toward Measuring and Maintaining the Zone of Proximal Development in Adaptive Instructional Systems,

Lecture Notes in Computer Science, 2002, Volume 2363/2002, 749-758

Adaptivity improves learning

Raw percent Correct (Pre and Posttest) Accuracy over attempted problems

Wayang is adaptive and adaptable to gender and can be designed for population subsets

(e.g., low achieving students; students with disabilities).

Important gender differences suggest girls make more productive use of Wayang

25

N=75 N=69 N=72

Neutrallevel

max

min

max

minN=36 N=35 N=36

GENDER DIFFERENCE: F(107,1)=4.82; p=0.03ANCOVA for Frustration Report

GENDER x CONDITION effect: F(213,3)=2.91; p=0.036Females getting JANE report lowest frustration, after accounting for baseline FRUSTRATION reported in

pretest survey (graph to the left).

Results: Reduced Frustration for Girls

During TutorBefore Tutor

26

N=77 N=71 N=65

ANCOVA for Confidence ReportGENDER x LEARNING-COMPANION effect: F(203,3)=2.57; p=0.05

Females getting LCs report highest confidence, after accounting for baseline confidence reported in pretest survey (graph to the left).

Neutrallevel

Before Tutor

N=35 N=35 N=36

GENDER DIFFERENCE: F(106,1)=7.8; p=0.006

Results: Improved Confidencefor Girls

During Tutor

27

Females have better perceptions of learning with the Wayang software, when characters are present.

N=96 High School StudentsRandomly assigned to each condition

ANCOVA for Posttest Mean Perception, pretest math score as a covariateGENDER X CONDITION: F(96,1)=11.8, p=0.001

N=36 N=30 N=30

N=46N=50

Results: Improved experience for girls

28

MID

DLE

SCHO

OL

High School 2N=84

High School 1N=159

Girls’ negative feelings for mathematics develop between middle and high school

N=230, Middle School 1

M F M F

N=230, Middle School 1

HIGH

SCH

OO

L

MATH LIKING CONFIDENCEwhile problem solving

FRUSTRATIONwhile problem solving

High School 2N=84

High School 1N=159

M F M F

High School 2N=84

High School 1N=159

M F M F

N=230, Middle School 1

29

Low achieving students slightly

decrease reports of confidence with NO

character

ANOVA for ConfidenceGain (Posttest Confidence - Pretest confidence):Significant interaction effect for MathAbility x Condition. F(91, 1)=4.4, p=0.04

Low achieving students increase their

confidence in problem solving

with characters

N=19 N=37

N=9 N=26

3.372.95 2.78 3.14

3.78 4.06 4.08 4.08

Results: Improved confidence for low achieving students

30

Low achievement students increase

reports of frustration with no characters

Low achievement students slightly

decrease reports of frustration with

characters

N=19 N=39

N=9 N=26

Means for pre/posttest frustration (N=93)

3.634.05

3.69 3.5

3.112.78 2.77 2.88

Results: Reduced frustration for low achieving students

Summary, Adaptive systems

Adaptive tutors attempt to keep students within a “zone of proximal development”

Adaptation improves learning.

Being adaptive over smaller “chunks” of similar problems (instead of the full set of problems) yields higher learning

Being “gentle” at increasing difficulty yields higher learning

5. Consider Executive Control

Neuroscience teaches us that the frontal cortex (the executive or CEO of the brain) provides the ability to distinguish a subtlety of expression. The frontal lobe is not well developed in teen agers. The immature brain produces reaction (not rational thought) and leads to impulsivity, risk-taking behavior.

How can you grow your frontal lobe? By placing effort on new tasks and creating complex and alternative pathways. How can you learn new skills? By reducing cognitive load (work the brain is doing) and avoiding distractions and multi-tasking.

Intelligent tutors help reduce cognitive load by proving easier problems, increasing student confidence and reducing frustration. Low achieving students and students with disabilities have particularly benefitted from such strategies.

33

Improvement on Easy/Hard ItemsLow achievement students learned more

Easy Questionspre and post

Hard Questionspre and post

Low math ability High math ability

Low math ability studentsimprove more than high achievement

students on both easyand hard items

Students of low math ability improved significantly more than high ability

students on hard items.

34

Closing the gap between IEP and typical students

The gap between IEP and non-IEP students decreases by 2% at posttest time

4%

6%

35% 41%51% 55%

Fantasy Adventures

Students solve real-world problems to enhance their conceptual understanding of mathematics. The adventures invite students to rebuild an orangutan infirmary (top left), to calculate the possibility of driving a jeep (top right) over a broken bridge and tp calculate the likelihood that loggers have illegally harvested logs (bottom). These are multiple steps within

novel problems.

Affective learning companions congratulate students on effort exerted and talk to them about their effort and learning. Mastery bars help students measure their learning. Recreation pages (video, music, animations) provide rewards.

The adventures invite students to rebuild an orangutan infirmary, to calculate the possibility of driving a jeep over a broken bridge and to calculate the likelihood that loggers have illegally harvested logs. These are multiple steps within novel problems

Detail Support for faculty

Provide instant feedback

to students on problems and progress

to faculty on individual students and

on problem difficulty for class and students

Diagnostic information about individual students

Student improved on each topic

One high achieving student

Diagnostic information about individual students

Student did not master the last three topics

One low achieving student

New insights into curriculum materials

Faculty identifies challenging (yellow) and really hard (red)

problems.

New insights into student skills

Faculty identifies each student’s low (red) and high

(blue) mastery levels.

• Teachers click on particularly difficult problems shown in red to see the problem and hints. They then review these problems with students.

• Traditional instructional systems often provide only a single data point on achievement without details about problems that were difficult for an individual student.