open-book examinations what it is and what it is not rocky k. c. chang department of computing 15...

Open-Book ExaminationsWhat it is and what it is not

Rocky K. C. Chang

Department of Computing

15 October 2003

Rocky K. C. Chang, Department of Computing2

Why not OX?

• Mine is 100% continuous assessment.• Mine requires a certain degree of

memorization.• The books would distract students during

exams.• I do not see the differences between closed

and open book exams.• It is much easier to design questions for

closed-book exams.• I am more comfortable with closed-book

exams.


Why OX?

• Students would buy books.• Students would spend more time on book

reading.• Students can stay away from the

memorization exercise before exam.• Students generally think that open-book exam

a very fair assessment.• Don’t have to worry whether students would

have found out the questions beforehand. • Don’t have to worry too much about

plagiarism during exam.


In my opinion, OX is …

• Not to make students happier– Some don’t.

• Not exam questions as usual– More in-depth, probing questions

• Not a decision towards the end– Communicated in the first class

• Not about the form and style of examination– About the teaching strategy

• Not about making information available during exam– About communicating expectation throughout the

course• Not about open BOOKS

– About openness in teaching and learning


Not to leave assessment alone

• Not assessment and teaching– Assessment as part of teaching

• Not assessment for the “final judgment”– Assessment for learning

• Not teach-and-then-assess– Teach-assess-teach-assess-teach- …

• Not assessment for the seen– Assessment for the unseen


Good vs bad assessments

• Assessment distinguishes students’ learning (normal distribution?).

• Assessment is congruent with teaching. • Assessment covers the syllabus and major

concepts.• Assessment covers a range of skills

(analytical, practical, communications, etc).• Assessment discourages bad learning habits.• Assessment effects good learning habits.


Why OX for me?

• Provide a thorough learning experience.• Two approaches: barrier removal and

enabling.• First presupposition: closed-book exam

is a barrier to thorough learning.– Instill insecurity: what if I miss ….– Encourage fact memorization and

discourage understanding.– Limit the depth of the assessment.– Inconsistent environment.


OX can

• Remove the insecurity by making it a nonsecretive issue.

• Encourage to spend time only on understanding the course material.

• Probe for more in-depth understanding as a result of the last item.

• Minimize the discrepancy between the usual working environment and the examination environment.


Enabling through

• Assignments and tutorials– Past examination and test questions– Let’s-work-together-work– Drill a deeper understanding– Encourage the study group to work on them.

• Dynamic interaction• Group-based learning• 1-2 mid-term tests• Hands-on class project


OX is everything

• Decided on OX before teaching.– Announce and explain it in the first class.– Guarantee no question recycling.

• My messages:– Communicate my expectations.– Show my seriousness in teaching.– Demonstrate my commitment in teaching.

• Enable students in the process


Impact on the students?

• Enjoy the exam more?• Improve the learning attitude?• Increase the class attendance?• More attentive to the lectures and tutorials?• Increase class participation?• Change from memorization to understanding?• Work harder?• Stay away from plagiarism?• Spend more time on reading the textbook?• Study more with other students?• Enhance the learning outcome?• Increase the interest in the subject?


The most recent survey: I

1. The examination questions require a thorough understanding of the subject materials.

2. The examination questions are difficult.

3. The examination questions can accurately assess my understanding of the subject materials.


The most recent survey: I

42.2

33.3

13.3

51.1

42.2

60

6.7

22.224.4

02.2 2.2

0 0 00

10

20

30

40

50

60

StronglyAgree

Agree Neutral Disagree StronglyDisagree

Thorough understanding Difficult Accurate assessment


The most recent survey: II

4. The solutions to the assignments and previous examination questions are helpful for the exam preparation.

5. The books and other materials that you brought are helpful.

6. The time given for the examination is adequate, i.e., given 1-2 more hours would not help.


The most recent survey: II

24.4

17.815.6

66.7

46.7

28.9

4.4

26.7

11.1

4.48.9

35.6

0 0

8.9

0

10

20

30

40

50

60

70

StronglyAgree


Solutions helpful Books helpful More time helpful


The most recent survey: III

7. I still prefer this kind of open-book examinations.

8. I think I have achieved my expected grade indicated in the first-class survey.

9. Overall, I enjoy taking this course.


The most recent survey: III

44.4

4.4

17.8

37.8

22.2

60

11.1

42.2

20

6.7

26.7

2.20

4.4

00

10

20

30

40

50

60

StronglyAgree


Prefer open-book Achieve my goal Enjoy the course


Other comments

• Always critical questions, so exciting, but frustrating.• Strongly agree on Q.1 and 2. This can really

distinguish very good students from average students.• Open book exam is very time consuming. Open book

exam works like assignment. Can consider close book exam with easier questions and less time.

• I found that I could learn a lot during this exam. Also, I realize what we should spend more time on it. Overall, I think this kind of examination is really interesting.

• Many questions are not easily be found in the book, really need to understand.


What have I gained?

• I have compiled a set of good problems.• I am more thorough in my teaching.• I have a better understanding of the students’

learning difficulties.• I have learned more during the time of

designing new questions.• I can gradually increase the depth of my

teaching.• I see my students put more effort in

understanding.• I have stayed alive in my teaching.


What does OX (not) achieve?

• Not important:– The books, notes, assignment solutions available

to the students

• Important:– Get rid of the fear of losing out to those good

memorizers.

• Very important– Encourage students spend to more time on

understanding.

• Utmost important– Cultivate a passion for thorough learning.


A final question

• Does our teaching produce students with an open mind?– Do they question?– Do they challenge?– Do they counter-propose?– Do they innovate?

• OX is an instrument to cultivate an open environment for teaching and learning.– Transparent teaching

The course promotes independent thinking rather than memorizing. This is the most special thing throughout

the two years of U-life.

- A student’s comment

open-book examinations what it is and what it is not rocky k. c. chang department of computing 15...

Documents

teaching assessment

learning slide

seen assessment

fair assessment

continuous assessment

chang department of

final judgment assessment

department of computing10