Writing Multiple Choice TestsPresentation to First Year Biology

Educators’ ColloquimProfessor Jeffrey Smith

University of Otago

The Basic ApproachStep One: The test blueprint – what do you

want to test? What is the content and what is the level of thinking you want to assess?

Is the test blueprint tightly related to the course outline/objectives?

Do the students know what you expect of them?

Levels of sophistication in thinking(Bloom’s taxonomy)KnowledgeComprehensionApplicationAnalysisSynthesisEvaluation

The Basic ApproachStep Two: Logistics. How much time is available for the test?What kinds of items will best fit your

purpose?Consider marking in addition to optimal

format from a measurement perspective. (Might a multiple choice format save you a lot of time down the road that could be used in more productive activities than marking essays?)

How many items and of what type? (Rule of thumb is a minute per item for MC, but this is easy to try out.)

The Basic ApproachStep 3: Align your items with your test

blueprint. What kinds of items are you going to use

where?Are you covering all of the content that is

important or just the content where it is easy to write items? (Beware of the tyranny of the readily quantifiable.)

The Basic ApproachStep 4: (Isn’t this fun?) Now you have your

test outlined and items assigned to the various components of the test. So what is next?

Draft the items.

Writing the Multiple Choice ItemWhat do you want to know about the student

with this item? What would a great item do here?

My approach tends to focus a lot on knowledge, comprehension, and application from Bloom’s taxonomy. Especially when writing MC items.

Writing the Multiple Choice ItemWrite the stem first. End it in a question mark.Pose a clear problem.Don’t worry about the choices right now.

Read the item. Think about what you are wanting to measure from your test blueprint. How good a match do you have?

Sample knowledge level item(What does it require?)What is the space between the end of the

axon of one neuron and the beginning of the dendrite of another neuron called?

  A. cortex B. corpus callosum C. glial cell D. synapse

Sample knowledge level item 2(What does it require?)Which lobe is associated with higher order

cognitive functioning, such as judgement and problem-solving?

A. frontal lobe B. parietal lobe C. temporal lobe D. occipital lobe

Sample comprehension level item (What does it require?)Which of the following best describes the transmission of information

from one neuron to another?   A. The axon of one neuron is attached to the dendrites of a

number of postsynaptic neurons, which lets electrical current be passed amongst neurons.

B. An electrical impulse down the axon of one neuron causes neurotransmitters to be sent across a synaptic gap and received by dendrites of another neuron.

C. The nucleus of the first cell divides, and the information contained in the DNA of that cell passes on to the DNA of the second cell through the synaptic cleft.

D. The axon of one neuron sheds its outermost glial cell, which is received by the postsynaptic neuron through the transmitters contained at the end of the dendrites of the receiving cell.

Sample comprehension level item 2(What does it require?)Which of the following best describes a social

constructivist perspective?  A. The process of learning moves from the

external to the internal. B. The process of learning is facilitated by

trying to make sense of the world. C. The process of learning involves taking in

information and effectively processing it. D. The process of learning is not linear, but

occurs sporadically over time.

Sample application level item (What does it require?)Simon has moved from the United States to New

Zealand. He plays softball well but has never played cricket. When invited to take a turn at bat, he tries to adapt the way he would swing with a softball bat to swinging a cricket bat. What would Piaget call this behaviour?

  A. Elaboration B. Assimilation C. Rehearsal D. Accommodation

Sample application level item 2(What does it require?)Mrs White is working with her Year 4 children on the properties of

magnets. She asks what a magnet does and Molly answers, “It can pull things.” Mrs White responds, “Let’s test that idea.” She gets out a cotton ball, a pencil, a piece of string, and a bit of clay and tries to pull them with the magnet. When none of them work, she asks what the problem is. Molly says, “That magnet is broken.” Steve says, “Maybe it isn’t a magnet.” What is Mrs White doing with the class?

  A. Trying to move the students into a formal operational way of

thinking. B. Trying to set up disequilibrium in the environment of the

students. C. Trying to encourage the students to recall information stored

in long term memory. D. Trying to get the students to adopt a hypothesis-testing

approach to science.

Drafting itemsKnowledge level

What is x called…Which of the following is associated with…Who did x…How many ….

Drafting itemsComprehension level

Which is the best description of… (Or, what does this describe?...)

What is the main advantage of…What kind of ….

(Or, according to theory x, which…)The major difference between x and y is….

Drafting itemsApplication level

Starting with the setting or vignette:Daniel is teaching his son Blake how to ride a bicycle. In the

beginning, he ran alongside Blake and helped him balance the bike. Then, he let go for a few moments to let Blake do the balancing himself. Now, Daniel is only holding on at the very beginning. What is Daniel engaged in?

Mr Chapman wants his chemistry class to learn the periodic table of elements. He wants the students to understand the basics of the molecular makeup of the elements and how they are related to one another. He realises that this is a large task for the students, so he is trying to come up with ways to help them in their learning. Which model of learning would be the most helpful here?

Right answersYour students will be nervous when they take

the test. Make sure your right answers are right and your wrong answers are wrong. Nobody likes “gotcha”!

Seriously, nobody does.Write the right answer first. Make sure it is a

clear and direct response to the question as posed, unambiguous, and correct.

Wrong answersWrong answers determine how difficult an

item is. Toby is using a new computer program to help him learn to read.

The program is set up so that whenever Toby sees a word he cannot pronounce, he can scroll over it with the mouse. The word is then pronounced out loud by the computer. What is this an example of?

  A. Mediation A. Meditation B. Transfer B. Trans Alpine C. Web-based inquiry C. Moose doots D. Scaffolding D. Scaffolding

Wrong answersWrong answers need to be:

Clearly wrongReasonable given the contextAbout the same length as right answersWritten as carefully and cautiously as right answersLetter C sometimesAs far away from being right as you want the item to be

difficult

The Basic ApproachStep 5 (Remember the steps?): Review, edit, revise

You are a scholar in the general area of biology, not test construction

I, on the other hand, am a scholar in the area of test construction

I still screw this up all the timeThus, we review, edit, and revise. Get a colleague to read

your test and give you feedback. (It won’t hurt too much.)Get your partner to do so (mine happens to be a scholar in

the area of testing, so that’s kind of a bonus – sometimes)Remember that your students will be nervous and will try

to overthink this stuff.

The Basic ApproachStep 6:Go back to your test blueprint and make sure

your test measures what you want it to.

The Basic ApproachStep 7:Take your test as if you were a student. This

helps for:TimingDetermining difficultyFinal adjustments

That’s It!(If I could have come up with five more steps,

you could become recovering psychometricians!)

Thanks!