michigan state university gathering and timely use of feedback from individualized on-line work....
TRANSCRIPT
Michigan State University
Gathering and Timely use of Feedback from Individualized
On-Line Work.
Matthew Hall
Joyce Parker
Behrouz Minaei-Bigdoli
Guy Albertelli
Gerd Kortemeyer
Edwin Kashy
Matthew Hall
Joyce Parker
Behrouz Minaei-Bigdoli
Guy Albertelli
Gerd Kortemeyer
Edwin Kashy
Michigan State University
Alfred P. Sloan Foundation
Michigan State University
Andrew W. Mellon Foundation
National Superconducting Cyclotron Laboratory
National Science Foundation
Acknowledgements
Michigan State University
How can we keep instructors informed about how their students are doing?
How can an instructor tell where their students are going wrong?
How can an instructor use this feedback to provide a better learning experience for their students?
Questions
Michigan State University
Physics 183
Class timetraditional lecture
demonstrations
lecture exercises
quizzes
Resources for studentsphysics learning center
office hours
First year Calculus based Physics
Michigan State University
First Year Calculus based Physics Course
Assessmentindividualized on-line homework with feedback
supplemental problems, and mid-term and final exams
Evaluationmid-term tests and final exam
Michigan State University
Course Management System
LON-CAPA
Individualized problemshighly randomized
Multiple response typesnumerical, classification, ranking, click-on-image, etc
Immediate feedbackmultiple attempts
Michigan State University
Class Statistics
Students (S = 400)
Total number of problems (P = 250)
Multiple attempts are allowed (A <= 15)
Problems can have many responses (R = 1..10)
Volume of data is significant– S x A x R for a given problem– S x P x A x R for all problems in the course
Michigan State University
Variety of data
Each student receives their own version of a problem– Different statements– Different pictures– Different answers
Michigan State University
Cutting through the complexity and volume of data
• General statistics of class performance on a single problem
• Feedback on responses to complex conceptual problems
• Feedback to highlight specific student errors
• Evaluation of resources for exams
• Evaluation of overall student performance
Michigan State University
General Statistics of Class Performance on a Single
Problem
For a given problem• number of students attempting
• number of attempts
• number of correct attempts
• number of attempts per correct answer
Michigan State University
Number of attempts per correct answer
Michigan State University
A Complex Conceptual Problem
Classify Statements14 statements grouped into 6
concepts
One statement is presented from each concept
Each statement must be classified as one of 5 options
All classifications must be correct
Michigan State University
Feedback on a Complex Conceptual Problem
Michigan State University
Final weekFollowing weekBefore discussion
Correct Answers by Concept
Michigan State University
Feedback to Highlight Student Errors
Vector addition problem
Classify statements based on comparison of two quantities
14 statements, 7 of which are the reverse of the other statements, grouped into 5 concepts
Michigan State University
Feedback to Highlight Student Errors
3. Time to row across for K is .. for C
4. Time to row across for C is .. for K
Michigan State University
Evaluation of Resources used on Exams
• Degree of difficultyDIFF = 1 – (earned points / possible points)
• Degree of discriminationClassify students by performance on other exam questionsUse same ratio as aboveDISC = (ratio for good students) – (ratio for bad students)-1 to +1 +1 Only good students earned points 0 everyone did the same -1 Only poor students earned points
Michigan State University
Mid-term Exam Statistics
Problem Exam 1 DIFF
Exam 1 DISC
Exam 2 DIFF
Exam 2 DISC
123456789
1011
0.200.160.400.440.320.000.230.210.360.400.25
0.400.310.400.570.380.000.330.240.630.590.31
0.700.130.190.410.520.180.700.570.550.87
0.240.200.310.570.110.260.360.350.580.14
Michigan State University
Evaluating Class Performance
• Cumulative homework scores
• Cumulative exam scores
Michigan State University
3-D grade plot
Michigan State University
Evaluating Class Performance
• Correlation is moderate (r=0.43)
• Most students do well on homework• Students think doing well on homework means
success in the class
• The instructor implemented a “grade extrapolator” to allow students to interactively pose their future performance and view the resulting grade
Michigan State University
• Presented tools for feedback
• The feedback reflects the complexity of the problems
• More feedback tools needed– different problem types– different presentations of student answers
Conclusion
Michigan State University
Extra Slide - Atwood Problem Analysis, Massless Pulley