Peer review in an economics writing course:
the Pen is still mightier than the SWoRD
Jennifer Imazeki
Department of Economics
http://economicsforteachers.blogspot.com
SWoRD
• Scaffolded Writing and Reviewing in the Disciplines– Students submit papers– Students review papers of 4-6 peers
• Comments and 7-point numeric rating
– Students ‘back evaluate’ reviews they receive• 5-point numeric rating
What makes SWoRD different?
• Student review and backevaluation scores grades for both writing and reviewing– Reviewing grade: “Accuracy” + “Helpfulness”
• Accuracy: Consistency compared to other reviewers• Helpfulness: Back evaluation ratings
– Writing grade: adjusted average of numeric ratings from reviewers
– Scoring algorithm to remove bias
Econ 449W: Economic Literacy
• “The goal of this course is to make you better economists by making you better writers… This course will focus on how to apply and use [economic] knowledge, by incorporating economic material and the economic way of thinking into writing for a more general audience.”
Econ 449W: Structure
2010:• 6 peer-reviewed papers,
exchanged in class• 1 reviewer for each
paper, plus instructor feedback
• No feedback for reviewers, participation points only
Potential benefits of SWoRD(vs. other peer review tools)
• Students read multiple papers, receive feedback from multiple reviewers
• Back evaluations provide incentive to take reviews seriously, give students feedback for improving reviews (without requiring instructor intervention)
• Rubric scores converted to grades with adjustment for bias
Econ 449W: 2010 vs. 2011
2010:• 6 peer-reviewed papers,
exchanged in class• 1 reviewer for each
paper, plus instructor feedback
• No feedback for reviewers, participation points only
2011:• 4 peer reviewed papers,
all online submission• 4-5 reviewers for each
paper, no/less instructor feedback on draft 1
• Writers back-evaluate reviewers, reviewing score based 50% on those back evaluations
Assignment timeline
First Draft 5 days
Reviews (5) 5 days
Final draft 5 days
Final reviews
(5)
Back reviews by Thurs
Back reviews by Wednesday
9pm Sunday 9pm Friday 9pm Wednesday 9pm Monday
Issues
• Lack of flexibility• Lack of transparency• Scoring algorithm questionable
Peer review in an economics writing course:
the Pen is still mightier than the SWoRD
Jennifer Imazeki
Department of Economics
http://economicsforteachers.blogspot.com