THAT’S THE QUEST!ON

SILVESTER DRAAIJER

DEVELOPING TEST QUESTIONS

Writing high quality questions is regarded as difficult

> A difficult task (T. M. Haladyna, 1997, p. 67; Osterlind, 1998,

p. 40)

> A unique skill (Downing & Haladyna, 2006, p. 11)

> An art (Popham, 1984, p. 40).

> Creativity is of essence too (Osterlind, 1998, p. 2; Rikers,

1988, p. 16; Vale, 1995).

> Especially developing plausible distracters is regarded as

being difficult (Roid & Haladyna, 1982), specifically the 3rd

distractor of a four-option multiple-choice question is hard to

develop and costs relatively much time (Mayenga, 2009)

Vrije Universiteit Amsterdam

VRAGEN MAKEN IS MOEILIJK

Kennis van het domein

Kennis van vraag- en toetsconstructie

Creatief zijn

Vrije Universiteit Amsterdam

VRAGEN MAKEN IS MOEILIJK

1. Thorough mastery of the subject matter being tested.

2. Possess a rational and well developed set of thinking processes to determine what is most important to test.

3. Must understand the individuals for whom the test or items are intended.

4. Must be a master of verbal communication.

5. Must be skilled in handling the special technology used for item writing.

James B. Olsen 2004

DEVELOPING TEST QUESTIONS - COMMUNIS OPINIO OF EXPERTS

> Is not a science, it‟s a craft

> Learning to develop good test questions takes 16-40 hours

of instruction-practice-feedback

There is little scientific basis for multiple-choice

item writing. Most item writing knowledge is

based on personal experience or wisdom passed

on from particular mentors.

Welch, Haladyna (2004)

Vrije Universiteit Amsterdam

“Item writing is an art. It requires an uncommon combination of special abilities. It is mastered only through extensive and critically supervised practice. It demands and tends to develop high standards of quality and a sense of pride in craftsmanship.”

“Item writing is essentially creative. Each item as it is being written presents new problems and new opportunities. Just as there can be no set formulas for producing a good story and a good painting, so there can be no set of rules that will guarantee the production of good test items.”

Robert L. Ebel 1951

Vrije Universiteit Amsterdam

“The process of selecting item ideas goes on

simultaneously with the process of inventing them. Skill in

item writing depends not only upon prolific inventiveness but

also upon discriminating judgment in the selection. In

selecting item ideas, the writer must consider their

appropriateness, importance, and probable discriminating

ability.”

Robert L. Ebel 1951

Vrije Universiteit Amsterdam

There is no automatic process for the production of item

ideas. They must be invented or discovered and in the

process, chance ideas and inspirations are very important.

It is possible, however, to stimulate these processes by

appropriate material.

James B. Olsen 2004

Vrije Universiteit Amsterdam

DIVERGEREN EN CONVERGEREN

9 Presentatie

Vrije Universiteit Amsterdam

QUESTION DESIGN AS PROBLEM SOLVING

Problem finding (Pretz et al. 2003): Recognizing, Defining, and Representing problems

Divergent thinking ability is necessary> Brainstorming

> Force-fitting (who, when, why, make larger or smaller, put to other use, change order, use adjacent concepts, find causes and effects etc.)

> Brain-writing

> Morfological approaches

Convergent thinking ability is necessary> Evaluate using criteria lists (the 31

guidelines by Downing and Haladyna, 2002!)

> Peer judgement Extended effort

Vrije Universiteit Amsterdam

VOORBEELD

In een stellingvraag moet u „vage‟ aanduidingen vermijden:

> Vermijd het woord kan of kunnen

> Maar wat als er staat: … behoort tot de mogelijkheden …

Vrije Universiteit Amsterdam

VOORBEELD

In de leerstof:

> De economische filosofie van het Mercantilisme is gericht op

het dienen van het nationaal belang zo mogelijk ten koste

van andere naties. Door het inzetten van

overheidsmaatregelen die export bevoordelen en import

tegengaan moet dat gerealiseerd worden.

De vraag

> Het mercantilisme is een economische filosofie. J/O

WEBSITE

http://docs.qdnatool.org

http://www.qdnatool.org

13

12-2-2013