1 introductory programming, criterion-referencing, and bloom raymond lister & john leaney...
TRANSCRIPT
1
Introductory Programming,Criterion-Referencing, and
Bloom
Raymond Lister & John Leaney
University of Technology, Sydney
Australia
2
McCracken, et al.
• Insert their graph here
• points to make ...– Multi-institutional!– weaker students flounder
• ... Leading to quote on next page ...
3
Buck & Stucki, SIGCSE 2001 Students who are required to write completeprograms as lab assignments are ...
“... overwhelmed, uncertain of how to begin, and grasping at the air … [leading] … to the self-destructive tendency to do experimental programming, where they just randomly throw things in to see if it helps ”.
4
McCracken, et al.
After their first “clean compile”, the weak students ...
“... are then surprised by what the program really does when presented with data”.
5
Norm-referenced Assessment
• Same assessment activity for all students
• Students graded “to a curve”– Aimed at the “middle student”?– Weak students flounder ...
• ... but pass anyway?
– Strong students do well ...• ... but were they challenged?
6
Criterion-Referenced Assessment
• Qualitatively different assessment activities for each grade.
• “Truth in sentencing”– Weak students learn something, without
holding back strong students.– Strong students are challenged without hurting
the weak students.
7
Grades & Bloom’s Taxonomy
“A” Bloom’s Synthesis & Evaluation level
“B”
“C” Bloom’s Knowledge & Comprehension level
The message of Bloom: test at all levels ofthe taxonomy.
Bloom’s Application & Analysis level
8
“C” and Bloom’sKnowledge/Comprehension (1)
• No requirement that a ‘C’ student manifest any sort of design skill.
• Lab Exam: Translate pseudo-code of a complete Java class into correct, readable code that compiles.– For example ...
9
For example ...• Declare a header for a public class, called
"BMIApplet", a subclass of Applet, which implements the methods prescribed by the "ActionListener" interface.
• Declare a header for a public constructor, which takes zero parameters.
• Declare an integer "BMI” and assign a value to "BMI", by calling "computeBMI", a method in this class.
10
“C” and Bloom’sKnowledge/Comprehension (2)
• The ability to read code, and demonstrate an understanding of what that code does.
• Tested by multiple choice questions.• Pass mark is a “mastery” 70%, not 50%.
– For example ...
11
For example ...The code for a class called “Counter” is given as follows (consider carefully the use of the “static” keyword).
public class Counter{
private static int count = 0;
public void increment( ){
++count;}
public int getCount( ){
return count;}
}
If the following code is executed:
Counter counter1 = new Counter( );Counter counter2 = new Counter( );counter1.increment( );counter2.increment( );int total = counter1.getCount( ) + counter2.getCount( );
the value in “total” is:
a) 0b) 1c) 2d) 3e) 4
12
Rote Learning for a “C” ?
• Approximately 30 “bottom passing” students with mark in the multiple choice exam of 70-77%
• On questions with previously seen code, 63-89% of bottom passing students got the question right, median 81%.
• On questions with unseen code, 63-85% of bottom passing students got the question right, median 78%.
• Conclusion: Not rote learners!!!
13
30% fail the multiple choice exam at their first attempt.
Truth in sentencing: If some students can’t even readcode, how can we expect them to write code? Why assess them by asking them to write code?
The message of Bloom: test at all levels ofthe taxonomy.
14
The Story So Far
“A” Bloom’s Synthesis & Evaluation level
“B”
“C” Completion of pseudo code; multiple choice questions
Bloom’s Application & Analysis level
15
“B” and Bloom’sApplication/Analysis (1)
Bloom distinguishes between comprehension and application as follows:
“A demonstration of comprehension shows that the student can use the abstraction when its use isspecified. A demonstration of application showsthat he (sic) will use it correctly, given an appropriatesituation in which no mode of solution is specified.”
16
“B” and Bloom’s Application/Analysis (2)
• No surprises here ...• Assessed traditionally, with an assignment.• “Pig World”• Third of class opt out and happy with “C”.
17
The Story So Far
“A” Bloom’s Synthesis & Evaluation level
“B”
“C” Completion of pseudo code; multiple choice questions
Traditional assignment
18
The “A” Part 1: Bloom’s Synthesis level
Bloom defines “Synthesis” as ...
"Skill in writing, using an excellent organization of ideasand statements" (p.169). (essays? programs?)
“... ability to write creatively a story, essay, or verse forpleasure, or for the entertainment or information of others" (p. 169).”
assessed by a project ...
19
The “A” Part 1: Bloom’s Synthesis level (cont.)
“Write a program. You decide what it does.”
To avoid a token effort, we specify:- at least 100 lines of code.- at least 4 classes- etc.
We want to let HD students to learn-from-play.Constructivism rules!
20
The “A” Part 2: Bloom’s Evaluation level
Bloom defines “Evaluation” as ...
"the making of judgments about the value, for some purpose, of ideas, works, solutions … ” (p.185).
Assessed by peer review.- I provide some vague evaluation criteria.
* clarity of code.* quality of user interface.
21
Conclusion
“A” Self-selected project; peer review.
“B”
“C” Completion of pseudo code;multiple choice questions
Truth in sentencing
The message of Bloom: test at all levels ofthe taxonomy.
Traditional assignment
22
Survey Results (1)
"My learning experience in this subject were interesting and thought provoking".
Agreed 69%
Neutral 18%
Disagreed 13%
23
Survey Results (2)
"I found the assessment fair and reasonable".
Agreed 64%
Neutral 30%
Disagreed 7%
24
Grade Distribution
Grade After After First MCQ Exam Second MCQ Exam
Fail 29% 10%
C 18% 37%
B 47% as before
A 6% as before