mcq testing benha faculty of medicine dermatology & andrology department undergraduate exam...

MCQ TESTING

Benha Faculty of Medicine

Dermatology & Andrology Department

Undergraduate Exam

January 2007 Time allowed 45 min

Why MCQs?

• Test a wide range of issues

• Not affected by a student's ability to write

• Fair marking

• Quickly marked by computer

• Computer gives analysis

of questions

• A bank of questions can be built up

• Can be used for quick revision

Weaknesses

of MCQ tests

• They do not test synthesis

• Time-consuming

• Student's must select from your alternatives

• Guessing may result in 20-25% right answers

• Attention to security

• Questions need to be pre-tested

Anatomy

of an MCQ

Impetigo is caused by:

1-Fungi

2-Parasites

3-Viruses

4-Bacteria

OPTIONSDISTRA-CTORS

ANSWER

STEM

Before writing

1- Find who's your audience

• Tailor the topics and difficulty of questions to the goals of the course

• Be sensitive to cultural and gender issues

2-Test for significant learning outcomes

• Knowledge  Recalling information

The outermost layer of the epidermis is the: 1-Malpighian layer 2-Basal layer 3-Horny layer 4-Granular layer

• Comprehension

Understanding the meaning of information

Syphilis in pregnancy can be treated by the following drugs EXCEPT:

1-Penicillin 2-Tetracyclins 3-Macrolides 4-Cephalosprins

• Application Solving a problem or making a decision

A diabetic patient with pemphigus

vulgaris should be given:

1-10-20 mg prednisolone/day

2-20 mg triamcinolone/day

3-Cyclophosphamide100-200mg/day

4-Acitratin 25 mg/day

3-Ask questions on

important aspects of

the topic

4-Focus on common student mistakes

e.g. griseofulvin in tinea versicolor

5-Do not give away answers to a previous question with a later question

6-Inform the examinees about the source of your questions

Writing The Stem

1-Determine the point to be tested

2- Avoid dealing with more than one problem

• In tinea capitis the best systemic treatment is:

• In eczematized tinea capitis the best systemic treatment is:

3-Avoid including words that do not contribute to the basis for choosing between options

Acne is a troublesome disease. Some drugs used in its treatment are dangerous during pregnancy e.g.:

Which of the following drugs is contraindicated in pregnant women with acne:

4-Include as much information in the stem and as little in the options as possible

A condition in which a man is unable to have an erection firm enough for intromission and copulatory movements till ejaculation is called:

ED is defined as:

5-Avoid negative questions

e.g. which one is NOT an antifungal?

• If this cannot be avoided, negative words should be: highlighted or in bold or in capitals

Which of the following drugs is NOT used in the treatment of ….

Writing The Options

1- Restrict the use of terms such as "always" or "never" in options (they are very rarely true)

2-Three or four well constructed options are enough. Generally, the minimal improvement to the exam due to adding a fifth option is not worth the effort to construct it

3- Avoid "All of the above"

(one recognized distractor eliminates it )

These drugs are antiscabitic :

1-Ectomethrine

2-Penicillin

3-Digitalis

4-All of the above

4- “None of the above" is acceptable if the question is factual

The pudedal artery is a branch from:

It is probably desirable if computation yields the answer

The dose of isotretinoin in a 60 kgm acne patient is :

5- Vary the location of the answer on as random a basis as possible. Students know that for some examiners the first option is rarely the answer

6-List the options in a logical order, e.g. if there are numbers the options should be arranged in ascending or descending order

7-Avoid grammatical cues i.e. one or more distractors don’t follow grammatically from the stem

A patient with itching that increases by night, a positive family history and penile itchy papule:

1-Have scabies 2-Has lichen planus

3-Is atopic

8-Options should be homogeneous

Crusts can be found in:

1-Ecthyma

2-Psoriasis

3-Lichen planus

4-ED

9-Distractors should be equally plausible to a weak candidate

10-Do not repeat information in all the options needlessly

Rifampicin in leprosy treatment :

1-Should be given as a 300mg dose once monthly

2-Should be given as a 600mg dose once monthly

3-Should be given as a 900mg dose once monthly

11-Have a colleague with expertise in the contents of the exam review the items

Marking

The Optical Mark Reading Scanner

• Students record their answers on a machine readable answer sheet

• These sheets are fed through an Optical Mark Reading Scanner that turns pencil marks into useable computer information

• If two or more circles are filled in on the answer sheet, the question will be marked 'wrong‘

• Specially written software controls the test scoring and analysis

Negative(formula)

marking:

To use or not to use?

Marks are subtracted every time an incorrect option has been selected

Theoretically, negative marking is intended to discourage guessing

The largest systematic study comparing right scoring and negative marking showed that using right scoring produced slightly higher scores than formula scoring

Disadvantages of negative scoring

• It takes examinees longer

• The directions are more complicated

• It might discourage answering based on good partial knowledge

• May actually penalise the better students who might be less likely to give educated guesses

Questions Evaluation

Does a question discriminate well

between examinees?

1-Questions answered correctly by nearly all examinees are useless for discriminating among students

2-Questions answered correctly by more than 85%of students have markedly reduced power to discriminate

3- Distractors that are not chosen by any examinees should be replaced

Measuring Discremination Index

ordering the test scores from lowest to highest ordering the test scores from lowest to highest

1-Arrange test scores from lowest to highest

12131415161718192021222324252627

2-Separate these scores into four equal parts

12131415161718192021222324252627

}

}

Lower quartile

Upperer quartile

3-Determine Ch and Cl

• Ch : No. of students who answered this question correctly in the high performing group

• Cl : No. of students who answered this question correctly in the low performing group

4-Calculate Discrimination Index for each group

Discrimination Index= (Ch - Cl)

Nh or Nl

Nh=No.of students in the high

performing group

Nl=No. of students in the low

performing group

The bigger the discrimination index for a question, the better its ability to dicriminate among students:

• Excellent: > 0.35

• Good: 0.35 - 0.25

• Acceptable: 0.25 - 0.15

• Deleted: < 0.1

Estimation of a Question Difficulty

• Easiness= Total No. of correct answers No. of students who took the test

• Difficulty = 1 – easiness

Too difficult <0.3 Too easy >0.7

Estimation of test difficulty is useful because:

• It can help determining passing score

• It gives experience in writing exams in the future