![Page 1: Multiple Choice Questions - USJPmedical.sjp.ac.lk/downloads/staff-development-materials/...MCQ, SEQ OSCE, OSPE, short case 360º Assessment, Portfolio Cognition Behaviour Miller’s](https://reader036.vdocument.in/reader036/viewer/2022062318/6102985ea1eacb2c1e2a1ce4/html5/thumbnails/1.jpg)
Multiple Choice Questions
![Page 2: Multiple Choice Questions - USJPmedical.sjp.ac.lk/downloads/staff-development-materials/...MCQ, SEQ OSCE, OSPE, short case 360º Assessment, Portfolio Cognition Behaviour Miller’s](https://reader036.vdocument.in/reader036/viewer/2022062318/6102985ea1eacb2c1e2a1ce4/html5/thumbnails/2.jpg)
Overview
• Why are MCQs used for assessment?
• How useful are they as assessment tools?
• How are MCQs constructed?
• What are the pros/cons of MCQs?
• Tips for constructing MCQs
![Page 3: Multiple Choice Questions - USJPmedical.sjp.ac.lk/downloads/staff-development-materials/...MCQ, SEQ OSCE, OSPE, short case 360º Assessment, Portfolio Cognition Behaviour Miller’s](https://reader036.vdocument.in/reader036/viewer/2022062318/6102985ea1eacb2c1e2a1ce4/html5/thumbnails/3.jpg)
• Assessment content should represent the curriculum
• Ideally the entire curriculum should be assessed
NOT PRACTICAL
Assessments…
![Page 4: Multiple Choice Questions - USJPmedical.sjp.ac.lk/downloads/staff-development-materials/...MCQ, SEQ OSCE, OSPE, short case 360º Assessment, Portfolio Cognition Behaviour Miller’s](https://reader036.vdocument.in/reader036/viewer/2022062318/6102985ea1eacb2c1e2a1ce4/html5/thumbnails/4.jpg)
CurriculumPopulation
Sample Assessment
Content sampling
![Page 5: Multiple Choice Questions - USJPmedical.sjp.ac.lk/downloads/staff-development-materials/...MCQ, SEQ OSCE, OSPE, short case 360º Assessment, Portfolio Cognition Behaviour Miller’s](https://reader036.vdocument.in/reader036/viewer/2022062318/6102985ea1eacb2c1e2a1ce4/html5/thumbnails/5.jpg)
• Content to be assessed: blood pressure measurement
• Learning outcome: be able to correctly perform bloodpressure measurement
• Assessment method: SEQ – Write a short note on bloodpressure measurement
Assessing curricular content
![Page 6: Multiple Choice Questions - USJPmedical.sjp.ac.lk/downloads/staff-development-materials/...MCQ, SEQ OSCE, OSPE, short case 360º Assessment, Portfolio Cognition Behaviour Miller’s](https://reader036.vdocument.in/reader036/viewer/2022062318/6102985ea1eacb2c1e2a1ce4/html5/thumbnails/6.jpg)
What is the problem here?
![Page 7: Multiple Choice Questions - USJPmedical.sjp.ac.lk/downloads/staff-development-materials/...MCQ, SEQ OSCE, OSPE, short case 360º Assessment, Portfolio Cognition Behaviour Miller’s](https://reader036.vdocument.in/reader036/viewer/2022062318/6102985ea1eacb2c1e2a1ce4/html5/thumbnails/7.jpg)
Does
Shows
Knows how
KnowsMCQ, SEQ
MCQ, SEQ
OSCE, OSPE, short case
360º Assessment, Portfolio
Cognition
Behaviour
Miller’s Pyramid
![Page 8: Multiple Choice Questions - USJPmedical.sjp.ac.lk/downloads/staff-development-materials/...MCQ, SEQ OSCE, OSPE, short case 360º Assessment, Portfolio Cognition Behaviour Miller’s](https://reader036.vdocument.in/reader036/viewer/2022062318/6102985ea1eacb2c1e2a1ce4/html5/thumbnails/8.jpg)
• Valid – broad sample of content
• Reliable – objective scoring: reproducible
• Feasible (ease of administration)
• Cost-effective (greater coverage of content per unit of testing
time)
• Can be constructed with relative ease (especially recall type
questions)
Why are MCQs commonly used?
![Page 9: Multiple Choice Questions - USJPmedical.sjp.ac.lk/downloads/staff-development-materials/...MCQ, SEQ OSCE, OSPE, short case 360º Assessment, Portfolio Cognition Behaviour Miller’s](https://reader036.vdocument.in/reader036/viewer/2022062318/6102985ea1eacb2c1e2a1ce4/html5/thumbnails/9.jpg)
• Higher chance of guessing
• Mainly tests factual recall
• Negative marking
• Though a candidate may correctly answer a “false” item, it does not mean that he/she knows the correct answer
Pitfalls
![Page 10: Multiple Choice Questions - USJPmedical.sjp.ac.lk/downloads/staff-development-materials/...MCQ, SEQ OSCE, OSPE, short case 360º Assessment, Portfolio Cognition Behaviour Miller’s](https://reader036.vdocument.in/reader036/viewer/2022062318/6102985ea1eacb2c1e2a1ce4/html5/thumbnails/10.jpg)
The ‘testwise’ student!
![Page 11: Multiple Choice Questions - USJPmedical.sjp.ac.lk/downloads/staff-development-materials/...MCQ, SEQ OSCE, OSPE, short case 360º Assessment, Portfolio Cognition Behaviour Miller’s](https://reader036.vdocument.in/reader036/viewer/2022062318/6102985ea1eacb2c1e2a1ce4/html5/thumbnails/11.jpg)
A Mock MCQ Paper
![Page 12: Multiple Choice Questions - USJPmedical.sjp.ac.lk/downloads/staff-development-materials/...MCQ, SEQ OSCE, OSPE, short case 360º Assessment, Portfolio Cognition Behaviour Miller’s](https://reader036.vdocument.in/reader036/viewer/2022062318/6102985ea1eacb2c1e2a1ce4/html5/thumbnails/12.jpg)
Stergomeba is an
a. Hetropote
b. Norgocyte
c. Optropin
d. Stermaca
e. Ablowa
Question 1
![Page 13: Multiple Choice Questions - USJPmedical.sjp.ac.lk/downloads/staff-development-materials/...MCQ, SEQ OSCE, OSPE, short case 360º Assessment, Portfolio Cognition Behaviour Miller’s](https://reader036.vdocument.in/reader036/viewer/2022062318/6102985ea1eacb2c1e2a1ce4/html5/thumbnails/13.jpg)
Acbetinosis is
a. Always found among antrogotes
b. Seen with Stpogytes
c. Usually leads to atropyoma
d. A benign condition
e. Never a cause of btrodetitis
Question 2
![Page 14: Multiple Choice Questions - USJPmedical.sjp.ac.lk/downloads/staff-development-materials/...MCQ, SEQ OSCE, OSPE, short case 360º Assessment, Portfolio Cognition Behaviour Miller’s](https://reader036.vdocument.in/reader036/viewer/2022062318/6102985ea1eacb2c1e2a1ce4/html5/thumbnails/14.jpg)
A patient was diagnosed as having Balatitis. What is the most
appropriate plan of management?
a. Conservative
b. Exercise
c. Dietary control
d. Treament with activated nortrol followed by brangotherapy
e. Drugs
Question 3
![Page 15: Multiple Choice Questions - USJPmedical.sjp.ac.lk/downloads/staff-development-materials/...MCQ, SEQ OSCE, OSPE, short case 360º Assessment, Portfolio Cognition Behaviour Miller’s](https://reader036.vdocument.in/reader036/viewer/2022062318/6102985ea1eacb2c1e2a1ce4/html5/thumbnails/15.jpg)
A 50 year old women is diagnosed to have menitritis. The
most appropriate combination/s of drug treatment for her
condition is/are
a. Antropin and Bactipan
b. Bactipan and Salmetin
c. Antropin and Dectin
d. Bactipan and Dectin
e. Dectin and Salmetin
Question 4
![Page 16: Multiple Choice Questions - USJPmedical.sjp.ac.lk/downloads/staff-development-materials/...MCQ, SEQ OSCE, OSPE, short case 360º Assessment, Portfolio Cognition Behaviour Miller’s](https://reader036.vdocument.in/reader036/viewer/2022062318/6102985ea1eacb2c1e2a1ce4/html5/thumbnails/16.jpg)
What are the symptoms of a second episode of infection?
a. Fever
b. Cough
c. Fatigue
d. Loss of appetite
e. All of the above
Question 5
![Page 17: Multiple Choice Questions - USJPmedical.sjp.ac.lk/downloads/staff-development-materials/...MCQ, SEQ OSCE, OSPE, short case 360º Assessment, Portfolio Cognition Behaviour Miller’s](https://reader036.vdocument.in/reader036/viewer/2022062318/6102985ea1eacb2c1e2a1ce4/html5/thumbnails/17.jpg)
• Format• Stem (statement)
• Items (responses)
………………………………………………………………………………………………………………………………………………………………………………….?a. …………………………..
b. …………………………..c. …………………………..d. …………………………..
e. …………………………..
T/F MCQs
![Page 18: Multiple Choice Questions - USJPmedical.sjp.ac.lk/downloads/staff-development-materials/...MCQ, SEQ OSCE, OSPE, short case 360º Assessment, Portfolio Cognition Behaviour Miller’s](https://reader036.vdocument.in/reader036/viewer/2022062318/6102985ea1eacb2c1e2a1ce4/html5/thumbnails/18.jpg)
• Avoid open statements. Always phrase the stem as aquestion.
• Avoid grammatical cues
• Avoid negative statements
“Each of the following is correct except…”
Tips for construction: Stem
![Page 19: Multiple Choice Questions - USJPmedical.sjp.ac.lk/downloads/staff-development-materials/...MCQ, SEQ OSCE, OSPE, short case 360º Assessment, Portfolio Cognition Behaviour Miller’s](https://reader036.vdocument.in/reader036/viewer/2022062318/6102985ea1eacb2c1e2a1ce4/html5/thumbnails/19.jpg)
•All options should be of equal length
eg.: A. Digoxin
B. Lidocaine
C. Phenytoin
D. Propranolol
E. Quinidine
• List in a logical order (numerically or alphabetically)
• Equal balance of true and false responses
Tips on construction: Responses
![Page 20: Multiple Choice Questions - USJPmedical.sjp.ac.lk/downloads/staff-development-materials/...MCQ, SEQ OSCE, OSPE, short case 360º Assessment, Portfolio Cognition Behaviour Miller’s](https://reader036.vdocument.in/reader036/viewer/2022062318/6102985ea1eacb2c1e2a1ce4/html5/thumbnails/20.jpg)
• Vary the order of T/F responses
• Avoid always having a true statement as the first response
• Avoid: ‘all of the above’, ‘none of the above’ responses
• Avoid using ambiguous terms – sometimes, frequently,rarely, usually
• Avoid absolute terms – never, always
Tips on construction: Responses
![Page 21: Multiple Choice Questions - USJPmedical.sjp.ac.lk/downloads/staff-development-materials/...MCQ, SEQ OSCE, OSPE, short case 360º Assessment, Portfolio Cognition Behaviour Miller’s](https://reader036.vdocument.in/reader036/viewer/2022062318/6102985ea1eacb2c1e2a1ce4/html5/thumbnails/21.jpg)
What are the causes of sinus tachycardia?
a) thyrotoxicosis
b) propranolol therapy
c) anxiety
d) exercise
e) sleep
Testing recall
![Page 22: Multiple Choice Questions - USJPmedical.sjp.ac.lk/downloads/staff-development-materials/...MCQ, SEQ OSCE, OSPE, short case 360º Assessment, Portfolio Cognition Behaviour Miller’s](https://reader036.vdocument.in/reader036/viewer/2022062318/6102985ea1eacb2c1e2a1ce4/html5/thumbnails/22.jpg)
A 45 year old woman with increasing “tiredness” and difficultyin swallowing has the following haematological indices
Hb 8.5 g/dl (14 +/- 2.0)
MCHC 29 g/dl (33 +/-2.0)
MCV 80FL (85 + /-8.0)
ESR 10mm (0-7)
True/false regarding this patient?a) Multiple myeloma is a likely diagnosisb) Koilonychia would be an expected findingc) Serum TIBC is likely to be increasedd) Bone marrow aspiration biopsy is clearly indicated
Testing application
![Page 23: Multiple Choice Questions - USJPmedical.sjp.ac.lk/downloads/staff-development-materials/...MCQ, SEQ OSCE, OSPE, short case 360º Assessment, Portfolio Cognition Behaviour Miller’s](https://reader036.vdocument.in/reader036/viewer/2022062318/6102985ea1eacb2c1e2a1ce4/html5/thumbnails/23.jpg)
• Format• Stem: clinical scenario, patient description, clinical
findings
• Lead-in question
• A series of response options
Single Best Response
![Page 24: Multiple Choice Questions - USJPmedical.sjp.ac.lk/downloads/staff-development-materials/...MCQ, SEQ OSCE, OSPE, short case 360º Assessment, Portfolio Cognition Behaviour Miller’s](https://reader036.vdocument.in/reader036/viewer/2022062318/6102985ea1eacb2c1e2a1ce4/html5/thumbnails/24.jpg)
1. ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
-----------------------------------------?
A……………..
B……………..
C…………….
D…………….
E…………….
SBR Outline
![Page 25: Multiple Choice Questions - USJPmedical.sjp.ac.lk/downloads/staff-development-materials/...MCQ, SEQ OSCE, OSPE, short case 360º Assessment, Portfolio Cognition Behaviour Miller’s](https://reader036.vdocument.in/reader036/viewer/2022062318/6102985ea1eacb2c1e2a1ce4/html5/thumbnails/25.jpg)
• Usually a clinical case commonly encountered in day to daypractice
• Not necessary for factual recall questions
Tips on construction: The stem
![Page 26: Multiple Choice Questions - USJPmedical.sjp.ac.lk/downloads/staff-development-materials/...MCQ, SEQ OSCE, OSPE, short case 360º Assessment, Portfolio Cognition Behaviour Miller’s](https://reader036.vdocument.in/reader036/viewer/2022062318/6102985ea1eacb2c1e2a1ce4/html5/thumbnails/26.jpg)
• Should clearly indicate how to answer the question
• Cover test
Tips on construction: Lead-in
![Page 27: Multiple Choice Questions - USJPmedical.sjp.ac.lk/downloads/staff-development-materials/...MCQ, SEQ OSCE, OSPE, short case 360º Assessment, Portfolio Cognition Behaviour Miller’s](https://reader036.vdocument.in/reader036/viewer/2022062318/6102985ea1eacb2c1e2a1ce4/html5/thumbnails/27.jpg)
• Only one clearly correct answer for the informed candidate.
• The distractors should be equally plausible to a weakcandidate.
• When ‘the best’ or ‘the most likely’ answer is sought thisshould be clearly stated in the lead-in.
Tips on construction: Responses
![Page 28: Multiple Choice Questions - USJPmedical.sjp.ac.lk/downloads/staff-development-materials/...MCQ, SEQ OSCE, OSPE, short case 360º Assessment, Portfolio Cognition Behaviour Miller’s](https://reader036.vdocument.in/reader036/viewer/2022062318/6102985ea1eacb2c1e2a1ce4/html5/thumbnails/28.jpg)
A patient with a history of ischaemic heart disease iscomplaining of central chest pain following laparoscopiccholecystectomy. A 12-lead ECG shows ST elevation in leads II,III and aVF.
Which coronary artery is most likely to have been occluded?
a. Left anterior descending artery
b. Left circumflex artery
c. Left coronary artery
d. Posterior interventricular artery
e. Right coronary artery
Example 1
![Page 29: Multiple Choice Questions - USJPmedical.sjp.ac.lk/downloads/staff-development-materials/...MCQ, SEQ OSCE, OSPE, short case 360º Assessment, Portfolio Cognition Behaviour Miller’s](https://reader036.vdocument.in/reader036/viewer/2022062318/6102985ea1eacb2c1e2a1ce4/html5/thumbnails/29.jpg)
From the list below, which of the following carcinomas of thelung is highly associated with exposure to asbestos?
a. Adenocarcinoma
b. Malignant mesothelioma
c. Large cell carcinoma
d. Small cell carcinoma
e. Squamous cell carcinoma
Example 2
![Page 30: Multiple Choice Questions - USJPmedical.sjp.ac.lk/downloads/staff-development-materials/...MCQ, SEQ OSCE, OSPE, short case 360º Assessment, Portfolio Cognition Behaviour Miller’s](https://reader036.vdocument.in/reader036/viewer/2022062318/6102985ea1eacb2c1e2a1ce4/html5/thumbnails/30.jpg)
General tips for constructing MCQs
• Questions should be clear and unambiguous
• Avoid taking statements directly from text books/lecture
notes
• Align questions to learning outcomes - in terms of content
and cognitive level
• Include questions enabling higher order thinking, not only
recall type questions
• Include questions with clinical relevance
General tips for constructing MCQs
![Page 31: Multiple Choice Questions - USJPmedical.sjp.ac.lk/downloads/staff-development-materials/...MCQ, SEQ OSCE, OSPE, short case 360º Assessment, Portfolio Cognition Behaviour Miller’s](https://reader036.vdocument.in/reader036/viewer/2022062318/6102985ea1eacb2c1e2a1ce4/html5/thumbnails/31.jpg)
Case, S.M., Swanson, D.B. & Becker, D.F. (1996) Verbosity, window dressing and red herrings: do they make a better test item?, Academic Medicine, 71, pp. S28–S30.
“MCQs which are well constructed assesstaxonomically higher-order cognitive processingsuch as interpretation, synthesis and applicationof knowledge rather than testing recall ofisolated facts”
![Page 32: Multiple Choice Questions - USJPmedical.sjp.ac.lk/downloads/staff-development-materials/...MCQ, SEQ OSCE, OSPE, short case 360º Assessment, Portfolio Cognition Behaviour Miller’s](https://reader036.vdocument.in/reader036/viewer/2022062318/6102985ea1eacb2c1e2a1ce4/html5/thumbnails/32.jpg)
General tips for constructing MCQs
• Create an acceptable balance of different levels of difficulty
• Ensure that items are psychometrically robust:
• Difficulty index
• Discrimination index
General tips for constructing MCQs
![Page 33: Multiple Choice Questions - USJPmedical.sjp.ac.lk/downloads/staff-development-materials/...MCQ, SEQ OSCE, OSPE, short case 360º Assessment, Portfolio Cognition Behaviour Miller’s](https://reader036.vdocument.in/reader036/viewer/2022062318/6102985ea1eacb2c1e2a1ce4/html5/thumbnails/33.jpg)
General tips for constructing MCQs
• Indication of the level of difficulty of the item
• Proportion of students who correctly answered the question
• Between 0.3– 0.8 (0.5– 0.75 for T/F questions)
Difficulty Index
Difficulty Index = Number of students who answered the question correctly
Total number of responses
![Page 34: Multiple Choice Questions - USJPmedical.sjp.ac.lk/downloads/staff-development-materials/...MCQ, SEQ OSCE, OSPE, short case 360º Assessment, Portfolio Cognition Behaviour Miller’s](https://reader036.vdocument.in/reader036/viewer/2022062318/6102985ea1eacb2c1e2a1ce4/html5/thumbnails/34.jpg)
General tips for constructing MCQs
• How well the item can discriminate between good and weak
students
• Student group divided into 3 groups based on overall test
score (upper and lower 27% and middle 46%)
Discrimination Index
Correct answers in upper 27% - Correct answers in lower 27%
27% of total number of responses
![Page 35: Multiple Choice Questions - USJPmedical.sjp.ac.lk/downloads/staff-development-materials/...MCQ, SEQ OSCE, OSPE, short case 360º Assessment, Portfolio Cognition Behaviour Miller’s](https://reader036.vdocument.in/reader036/viewer/2022062318/6102985ea1eacb2c1e2a1ce4/html5/thumbnails/35.jpg)
General tips for constructing MCQs
• Between – 1.00 to +1.00.
• Acceptable >0.3. Higher the better.
• Items with negative values must be revised or removed
Discrimination Index