Physics Education ResearchThe University of Edinburgh

The benefits of giving feedback: investigating

patterns of student exchanges

over the PeerWise system

Alison Kay and Judy Hardy

Phys. 1A GGA Chem. 1B2011-12 2012-13 2011-12 2012-13 2011-12 2012-13

Students 172 275 215 276 155 141% of mark 4 4 4 4 3 3Authored 3 2 4 4 2 2Answered 15 10 20 20 10 10Rate & comment 9 6 5 5 6 6

Total comments 5058 3923 5910 12166 4685 2798

Implementation of PeerWise

1 Nonsense; totally irrelevant; symbols; undecipherable

2 Clear reply to a previous post – no reference to question or content

3 Basic statement that got question correct or incorrect; hit wrong button

4 Surface comments on questionSurface comments on answer

5 Deeper comments on question and specific features Deeper comments on own understanding

6 More evaluative - specific about HOW to improve the question or giving more detail WHY the question was good or notPossibly suggesting a different way of working out the answer – showing examples or calculations

7 As 6 but much more detail – how to take question or explanation further. More in-depth discussion

Comment coding

Comments examples1 “Why oh why. Sigh.”

2 “Glad I could help ;D”

3 “Haha! misread the question :P”

4 “Wow! What a question! Superb effort!”

5 “I too was caught out with the area of a sphere instead of a circle. Good explanation and good alternatives in multiple choice.”

6 “I'm not sure if your solution is correct. We can use a trajectory equation only if the initial and final heights are the same !!!”

7 “Nice question, but bad distractors (imo). I didn't expect an inelastic collision and just worked with energy conservation (m1gh1 = (m1+m2)gh2) and got 1.76 m. Would have totally thrown me off if that was an available answer.”

Which activity is most strongly associated with exam performance?

DV: Exam scoreIV: Number of answers or comments made

Physics 1A 2012-13Std. Beta

No. answers submitted .248** Total no. comments .239***

No. comments >3 .240***

No. comments >4 .281***

Physics 1A 2011-

12

Physics 1A 2012-

13

Chem 1B 2011-12

Chem 1B 2012-13

GGA 2011-12

GGA 2012-13

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

Plot of standardized coefficients from regression analyses

Answers

All com-mentsComments >3

Comments >4

Course

Sta

nd

ard

ized

Beta

Example: Physics 1A 2012-13Is writing quality comments associated with higher

exam performance when controlling for other factors?

Dependent Variable: Exam scoreIndependent Variables: No. comments coded > 4

Pre-test mark (FCI) Scottish Major Gender

Building the modelR2 Adjusted R2 F Value P (one tailed)

.287 .274 31.415 0.000

IndependentVariables

Beta Standardized Beta

P (one tailed)

Intercept 42.289 0.000Comments > 4 .346 .182 0.000

FCI .371 .375 0.000Scottish -6.599 -.203 0.000

The model predicts that each new high quality comment, is associated with a 0.35% increase in exam score.

The minimum 6 comments associated with a 2% increase .The mean of 14 comments associated with a 5% increase.

Physics 1A 2011-12

Physics 1A 2012-13

Chem 1B 2011-12

Chem 1B 2012-13

GGA 2011-12

GGA 2012-13

-0.4

-0.2

0

0.2

0.4

0.6

0.8

Combined Plot of standardized coefficients from regression analyses

Comments >4

Pre-score

Scottish

Major

Gender

Course

Sta

nd

ard

ized

Beta

ConclusionsTotal engagement with PeerWise far exceeded minimum requirements

Across all disciplines, writing meaningful comments has a significant, positive association with exam performance when controlling for other known influential factors

Especially interesting given that PeerWise requires minimal instructor intervention; that PeerWise is only worth ~4% of the total course mark; and that providing feedback is only one aspect of the PeerWise assessment