Self-diagnosis, Scaffolding and Transfer: A tale of two

ProblemsA. Mason1, E. Cohen2, C. Singh1

and E. Yerushalmi21University of Pittsburgh

2Weizmann Institute of Science, Rehovot, Israel

PERC 2009

Research Design

Goal: recitation sections deliberately prompted to self-diagnose, given varying levels of support

– Who will perform the best?

1st stage: Students attempt a quiz problem, 2nd stage: Intervention-Self-diagnosis with alternative supports

Control-no SD TA discuss solution, (A)

TA outline +diagnosis Rubric (B)

Sample Solution(C)

Text+ note Book (D)

3rd stage Post: Solving paired quiz problem

100 students

31 students 28 students 24 students

First-semester algebra-based introductory physics1 instructor, 2 TAs

1) What are students able to diagnose if deliberately prompted to self-diagnose given alternate supports?

2) What is the effect of SD on consecutive problem solving?

Assumption: high-performers are the ones which learn from their mistakes. If we can get low-performers to learn from their mistakes,

3) Do we reduce the gap between low- and high-performers?

Research Rationale

Fred Flintstone just got off work, and exits in his usual way, sliding down the tail of his dinosaur and landing in his car (see Figure). Given the height of the dinosaur (h=10 m), it's not hard to calculate his speed v as he enters his vehicle.Conservation of energy yields the following equation: mgh=1/2 mv2, where m=100 kg is Fred's mass and v is his speed. Algebraic manipulation yields v=sqrt(2gh)=14 m/s. Judging from the picture taken in Figure 1E, the angle at which Fred enters the car is approximately 450. (a) If the mass of the car is M=200 kg, find the speed with which Fred is driving in the last frame (Figure 1F), assuming he hasn't used his feet to pedal. (Remember also that there are no fossil fuels since there are no fossils yet.) (b) Assuming that there is no friction or air resistance, determine the maximum height H that Fred and his car can travel without extra pushing.

You are helping a friend prepare for the next skate board exhibition. Your friend who weighs 60 kg will take a running start and then jump with a speed of 1.5 m/s onto a heavy duty 5 kg stationary skateboard.Your friend and the skateboard will then glide together in a straight line along a short, level section of track, then up a sloped concrete incline plane. Your friend wants to reach a minimum height of 3 m above the starting level before he comes to rest and starts to come back down the slope. Knowing that you have taken physics, your friend wants you to determine if the plan can be carried out or whether he will stop before reaching a 3 m height. Do not ignore the mass of the skateboard.

Boy jumping on skateboard going over a inclined plane –what height will he reach? (i.e. solve for max height)

Quiz 7 and its Post (midterm III)Quiz 6 and its Post (midterm II)

A family decides to create a tire swing in their back yard for their son Ryan. They tie a nylon rope to a branch that is located 16 m above the earth, and adjust it so that the tire swings 1 meter above the ground. To make the ride more exciting, they construct a launch point that is 13 m above the ground, so that they don't have to push Ryan all the time. You are their neighbor, and you are concerned that the ride might not be safe, so you calculate the maximum tension in the rope to see if it will hold. (a) Where is the tension greatest? (b) Calculate the maximum tension in the rope, assuming that Ryan (mass 30 kg) starts from rest from his launch pad. Is it greater than the rated value of 750 N? (c) Name two factors that may have been ignored in the above analysis, and describe whether they make the ride less safe or more safe.

A friend told a girl that he had heard that if you sit on a scale while riding on a roller coaster, the dial on the scale changes all the time. The girl decides to check the story and takes a bathroom scale to the amusement park. There she receives an illustration (provided), depicting the riding track of a roller coaster car along with information on the track (the illustration scale is not accurate). The operator of the ride informs her that the rail track is smooth, the mass of the car is 120 kg, and that the car sets in motion from a rest position at the height of 15m. He adds that point B is at 5m height and that close to point B the track is part of a circle with a radius of 30m. Before leaving the house, the girl stepped on the scale which indicated 55kg. In the roller coaster car the girl sits on the scale. Do you think that the story she had heard about the reading of the scale changing on the roller coaster is true? According to your calculation, what will the scale show at point B?

Girl on rollercoaster going over a circular bump – at peak of bump, how does weight change on a scale?

1st study 2nd studyNon-Conventional problem, Paired midterm problem given a few days after self-diagnosis

Conventional problem, Paired midterm problem given a month after self-diagnosis

Both pairs exhibit similar general procedure (principles/intermediate variables), different

surface features

I-quiz(grade of

quiz solution given by research

er)

S-quiz(grade of

quiz solution as diagnosed

by student)

I-SD (researcher’s grade of

the diagnosis

done by the student)

Invoking Appropriateprinciples

EC, MC, Newton second law…

Justification Justify EC and MC…

Compactness

Applying EC, MC, Newton second law…

Desc-ription

DrawingKnowns

FBD, masses & velocities before and after collision, velocities and heights or Ek and Ep before and after going up an incline…

plan Target + intermediate variables

vah…

checking Units, limit case

Total Scores (Phy. Score, Pres. Score)

Pre

senti

ng r

easo

nin

gPhysi

csAnalysis Tools-Analysis Rubric

Generic Specific

Valid: focus on students’ diagnostic ability

Versatile: generic/specific

Reliable: Inter-rater reliability > 80%

Research questions and expectations:

Research Questions

Inter group comparison: Given alternative supports, how will external support

affect self-diagnosis? how will it affect consecutive

problem-solving

Intra group comparison: How does the self-diagnosis affect the gap between low and high achieving

students? Do low achievers succeed

in diagnosing themselves? Do they perform better in

consecutive problem-solving?

Expectations More external support better self-diagnosis better performance on consecutive problem-solving

The self-diagnosis task will be successful in reducing the gap between low and high achieving students

Reservations:If self-diagnosis is NOT MEANINGFUL

Then between-group differences in self-diagnosis performance won’t be reflected in consecutive problem-solving

The gap between low and high achievers will remain the same in consecutive problem-solving

What is a MEANINGFUL self-diagnosis?THEORY (Chi, 2000):THEORY (Chi, 2000): Students learn from solved examples by providing self-explanations. The student is expected:

SamplSample e

SolutiSolutionon

Student’Student’s s

SolutionSolution

COMPARISON

Sig

nifi

can

t

diff

ere

nce

Student’s mind:

“Flawed Mental Model”

Instructor’s mind:

“Scientific Mental Model”

CONFLICT

Self-repairing flawed mental model

a) to compare and realize significant differences between the two solutions;

b) to acknowledge conflicts between their mental model and the sample solution, leading to self-repair of a flawed mental model.

1. “Meaningful intervention”: Students’ scoring on the diagnosis of their mistakes is independent of their prior knowledge, and involves self-repairing their mental modelOnly in this case do we expect better self-diagnosis to lead to better performance on consecutive problem-solving, thus reducing the gap between low and high achievers.

VS.

2. “Superficial intervention“: Students’ scoring on the diagnosis of their mistakes is independent of their prior knowledge, yet students do not involve self-repairing their mental model

3. “Weak intervention“: Students’ scoring on the diagnosis of their mistakes depends on their prior knowledge.

How does a NON-MEANINGFUL self diagnosis differ from a MEANINGFUL one?

Expectations - intra group comparison

Correlation Control

InterventionMeaningf

ul(Supports weak

students in improving their SD performance.Involves meaningful learning leading to transfer).

Superficial

(Supports weak students in improving their SD performance,However, does not involve meaningful learning leading to transfer).

Weak (Does not support weak students to improve their SD performance)

Pre vs. SD N/A N/S N/S Positive

SD vs. Post N/A Positive N/S Positive

Pre vs. Post Positive N/S Positive Positive

Inter group findings

Intervention

Outline + Rubric (B)

Sample solution (C)

Minimal guidance (D)

1st study (non

conventional problem)

SD mean (Std. Err) 0.73 (0.05) 0.57 (0.05) 0.24 (0.06)

P value B>C>D (ANCOVA based on quiz physics)

Post mean (Std. Err)

0.53 (0.05) 0.33 (0.05) 0.47 (0.06)

P value C differs from B and D (ANCOVA based on quiz physics)

2nd study (conventional problem)

SD mean (Std. Err) 0.56 (0.06) 0.62 (0.06) 0.61 (0.05

P value No difference between groups (ANCOVA based on quiz physics)

post mean (Std. Err)

0.66 (0.04) 0.72 (0.06) 0.76 (0.04)

P value No difference between groups (ANCOVA based on quiz physics)

How do alternative external supports affect self-diagnosis?

External support makes a difference in a non conventional problem, but not in a conventional one. e.g. Text and notes are enough for group (D) to perform SD in quiz 7, but not in quiz 6, where to self-diagnose students needed sample solution and rubric.

How do alternative external supports affect consecutive problem-solving?

It seems it doesn’t: e.g A) SD average grade for group B in quiz 6 was much better than that for the other groups, but their post average was comparable to group D. B) SD average grade for group D in quiz 7 was the same as group C, and slightly better that that for group B (not significantly). D got the highest average in their post, although this average was comparable to groups B and C.

The inter group findings suggest:

Intra group findings, 2nd study group D Control

A, A’intervention

Weak Superficial

Meaningful

Correlation Pre-SD

N/A N/S

Correlation SD-Post

N/A Positivecorr=0.53, p value<0.05)

Correlation Pre-Post

Positive0.44/0.35p<0.05

N/S

Intra group findings, 1st study all groups, 2nd study all groups but

D Control

A, A’intervention

Weak Superficial

Meaningful

Correlation Pre-SD

N/A N/S N/S

Correlation SD-Post

N/A N/S

Correlation Pre-Post

Positive0.44/0.35p<0.05

N/S

?

The intra group findings suggest: Group D, quiz 7 All other attempts, all

groups

D performs a meaningful diagnosis - Pre/Post correlation is not significant (midterm performance is independent of prior knowledge), while it is positive for control

- SD/Post correlation is positive (midterm performance depends on SD performance)

PUZZLE – no match to expectations!- On one hand, the intervention seems to be meaningful (Pre-post positive for control & none for intervention groups)

- On the other hand, it seems superficial, as for most groups there is no correlation SD-Post.

Self diagnosing with minimal help (D, 2nd

study)

=

D had to struggle to find a related sample solution in the text, which they could in the 2nd study (conventional) meaningful diagnosis

Possible resolution for puzzle group D

Meaningful SD Meaningful SD

Possible resolution for puzzle groups B and C

Self diagnosing with maximal

help (B,C)

It is hard to differentiate between meaningful and superficial diagnosis as the sample solution allows to diagnose “without struggling”

Meaningful SD Superficial SD ???

Possible resolution for puzzleWhat is unique for group D?

Meaningful self diagnosis involves [Chi]:Stage a) comparing and realizing significant differences between the two solutions; Stage b) acknowledging conflicts between a flawed mental model and the sample solution;

Student rarely make explicit remarks reflecting stage b, thus, the question is whether we can conclude from remarks reflecting stage a - difference between the two solutions - that a meaningful self diagnosis took place,

We claim that we can do so only for students who received minimal support for SD.

Possible resolution for puzzleWhat is unique for group D?

Students in groups B and C could compare their solutions to the sample solution provided and with minimal cognitive engagement state: "I did not do this equation".

Yet, in group D, students who were able to diagnose must have

been cognitively engaged as they had to search on their own

for a solution related to the problem they were trying to Diagnose.

Therefore, for group D the grade indeed indicates how meaningful the diagnosis is.