educationalassessment
DESCRIPTION
EducationalAssessment. Classroom MCQs Clinical OSCE. Objectives. By the end of seminar learners will be able to: Define assessment Overview of assessment and educational assessment Explain attributes of assessment - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: EducationalAssessment](https://reader036.vdocument.in/reader036/viewer/2022062423/568146ff550346895db43c96/html5/thumbnails/1.jpg)
EducationalAssessment
Classroom MCQs
Clinical OSCE
![Page 2: EducationalAssessment](https://reader036.vdocument.in/reader036/viewer/2022062423/568146ff550346895db43c96/html5/thumbnails/2.jpg)
Objectives By the end of seminar learners will be able to: Define assessment Overview of assessment and educational assessment Explain attributes of assessment Describe multiple choice questions (MCQs) Discuss the steps of designing MCQs. Identify advantages and disadvantages of MCQs Define OSCE Discuss the process of OSCE Converse approaches/models of OSCE Debate on OSCE Discuss Ethical issues in OSCE Know the views of students and faculty for OSCE
![Page 3: EducationalAssessment](https://reader036.vdocument.in/reader036/viewer/2022062423/568146ff550346895db43c96/html5/thumbnails/3.jpg)
Definition of Assessment Assessment comes from the old French
word assessor, meaning to sit by as an assistant judge or guide.
The process of collecting information on student achievement and performance. Provides information for basis of sound decision making regarding teaching and learning.
http://www.aac.ab.ca/final
![Page 4: EducationalAssessment](https://reader036.vdocument.in/reader036/viewer/2022062423/568146ff550346895db43c96/html5/thumbnails/4.jpg)
Cont… “The process of assessment is to
gather, summarize, interpret, and use data to decide a direction of action”
(Bastable 2003)
“Begins with educational values, works best when program has clear purposes…is ongoing and meets the responsibilities to students and public”
(Billings 2003)
![Page 5: EducationalAssessment](https://reader036.vdocument.in/reader036/viewer/2022062423/568146ff550346895db43c96/html5/thumbnails/5.jpg)
Educational Assessment “Is the process of documenting, usually
in measureable term, knowledge, skills, attitudes, and beliefs”
“It focus on the individual learner, the learning community, the institution, or the educational system as a whole”
http://en.wikipedia.org/wiki/Assessment
![Page 6: EducationalAssessment](https://reader036.vdocument.in/reader036/viewer/2022062423/568146ff550346895db43c96/html5/thumbnails/6.jpg)
Overview of Assessment
Assessment
why
where How
what
Whom
when
Of What
![Page 7: EducationalAssessment](https://reader036.vdocument.in/reader036/viewer/2022062423/568146ff550346895db43c96/html5/thumbnails/7.jpg)
Assessment
What? process of gathering… Why?
Measure academic achievement Identify areas for improvement Predict future performance
![Page 8: EducationalAssessment](https://reader036.vdocument.in/reader036/viewer/2022062423/568146ff550346895db43c96/html5/thumbnails/8.jpg)
Of what? Domains Cognitive (Knowledge)
Affective (Attitude)
Psychomotor (Skills)
Content Facts and principles Processes Signs and symptoms Clinical methods Techniques Management Etc….
![Page 9: EducationalAssessment](https://reader036.vdocument.in/reader036/viewer/2022062423/568146ff550346895db43c96/html5/thumbnails/9.jpg)
How? Written tests
Essay True false MCQs MCIs Viva/oral Practical Long case/short case Osce Projects
![Page 10: EducationalAssessment](https://reader036.vdocument.in/reader036/viewer/2022062423/568146ff550346895db43c96/html5/thumbnails/10.jpg)
How?
Shows how
Does
Knows how
Knows Factual test: MCQs, Essay, Oral
(Clinical) Context based testsMCQ, essay, oral
Performance assessment in vitroOSCE, Standardized-patient based test
Performance assessment in vivoUndercover SPs, Video, Logs only in real world of practice
(Miller 1990)Both areas tested by traditional written exams….
![Page 11: EducationalAssessment](https://reader036.vdocument.in/reader036/viewer/2022062423/568146ff550346895db43c96/html5/thumbnails/11.jpg)
When?
Continuous assessment
End of yearassessment
Strength Multiple points of observation can lead to greater confidence in judgment
Easier to define specific objectives and structured test content
Limitations
Assessor bias Busy facultyMay not observe
More artificial
![Page 12: EducationalAssessment](https://reader036.vdocument.in/reader036/viewer/2022062423/568146ff550346895db43c96/html5/thumbnails/12.jpg)
Where? Testing in situations similar to those in training and practice, enable to make justifiable decisions
![Page 13: EducationalAssessment](https://reader036.vdocument.in/reader036/viewer/2022062423/568146ff550346895db43c96/html5/thumbnails/13.jpg)
Assessment TypesFormative Summative
Strength Can identify weaknesses and further guide student learning
Can provide a clear diagnostic statement about a student competence
Limitations
May not be taken seriously by non self-directed learners
Usually occur too late to have any meaningful impact on learning
![Page 14: EducationalAssessment](https://reader036.vdocument.in/reader036/viewer/2022062423/568146ff550346895db43c96/html5/thumbnails/14.jpg)
Curricular attributes include Program and course objectives Critical learning experiences Learning outcomes
Faculty attributes comprise Level of content knowledge attitude about teaching Instructional skills Appropriate knowledge of teaching learning theories
Students’ attributes consist of Knowledge and skills Cognitive abilities Learning styles Motivation to achievement (Billings 2005)
Attributes of Assessment
![Page 15: EducationalAssessment](https://reader036.vdocument.in/reader036/viewer/2022062423/568146ff550346895db43c96/html5/thumbnails/15.jpg)
How to develop effective Assessments?
Should be RELIABLE and VALID
Reliability: A reliable assessment is one that provides consistent “scores”.
Validity: An assessment is valid when it accurately measures what is to measure.
![Page 16: EducationalAssessment](https://reader036.vdocument.in/reader036/viewer/2022062423/568146ff550346895db43c96/html5/thumbnails/16.jpg)
Multiple Choice Questions. MCQs Consists of twoparts Stem
Can be a question or an incomplete statement, scenario, problem
Several options/ distracters But to choose the
best response
![Page 17: EducationalAssessment](https://reader036.vdocument.in/reader036/viewer/2022062423/568146ff550346895db43c96/html5/thumbnails/17.jpg)
Steps in Designing MCQsStem
Clear single problem Content should be common Independent of all others in the test Information in one stem should not give clue to
another item Scenarios incorporated carefully Use of action verbs of Bloom taxonomy (1956)
according to cognitive level of students Knowledge, comprehension, application, analysis
(Billings, 2005)
![Page 18: EducationalAssessment](https://reader036.vdocument.in/reader036/viewer/2022062423/568146ff550346895db43c96/html5/thumbnails/18.jpg)
Steps in Designing MCQsOptions
Grammatically consistent with stem to avoid giving clues to right one
Arrangement order, alphabetical, numerical
Having same length Give only one best option, to which
all faculty agree Should not use all of above, none of
above
![Page 19: EducationalAssessment](https://reader036.vdocument.in/reader036/viewer/2022062423/568146ff550346895db43c96/html5/thumbnails/19.jpg)
Examples of MCQs Knowledge1. A patient’s father died of Huntington’s
chorea. What are the chances that pt will have the disease?
a) 25%b) 50%c) 75%d) 100%
![Page 20: EducationalAssessment](https://reader036.vdocument.in/reader036/viewer/2022062423/568146ff550346895db43c96/html5/thumbnails/20.jpg)
Comprehension
2. A pt resists the nurse's attempts to get him out of bed. If this attempt is without pt’s approval, what legal charge could nurse face?
a) Assaultb) Batteryc) Negligenced) Tort
![Page 21: EducationalAssessment](https://reader036.vdocument.in/reader036/viewer/2022062423/568146ff550346895db43c96/html5/thumbnails/21.jpg)
Application 3. On the 1st post-op day after an open reduction
and internal fixation (ORIF) of the tibia with application of long leg cast, the pt begins complaining of severe pain uncontrolled by his scheduled pain med. What would be the priority nursing intervention?
a) Administer a narcotic bolus as orderedb) Assess the neurovascular status of legc) Lower the leg to increase arterial flowd) Raise the leg to decrease venous return
![Page 22: EducationalAssessment](https://reader036.vdocument.in/reader036/viewer/2022062423/568146ff550346895db43c96/html5/thumbnails/22.jpg)
Analysis 4. A 70 yrs old woman is admitted to the
hospital with a diagnosis of dehydration. Vital signs are stable. Serum sodium level is 165 mmol/L. Which of the fluid replacement is most likely to be administered?
a) Hypertonicb) Hypotonicc) isotonic
![Page 23: EducationalAssessment](https://reader036.vdocument.in/reader036/viewer/2022062423/568146ff550346895db43c96/html5/thumbnails/23.jpg)
What makes MCQs “Good”? Alignment: to content, learning outcomes/goals, Importance: testing details is a waste of
resources; concentrate on fundamental concepts Clarity: the intent of the task and the meaning of
the options must speak for themselves and be interpreted in the same way by all examinees
Item construction: well-constructed items can be answered successfully by examinees who have the knowledge or skill you are testing, and NOT by examinees who are lacking the skill or knowledge
![Page 24: EducationalAssessment](https://reader036.vdocument.in/reader036/viewer/2022062423/568146ff550346895db43c96/html5/thumbnails/24.jpg)
MCQs Advantages Large content in single
test
Scored easily and objectively
Measure understanding at numerous cognitive levels in taxonomy category
Disadvantages Time consuming in
designing for faculty More time needed by
students to read and understand
Differentiate against creative, verbal student
Writing style affects scores
Difficult to write MCQs at synthesis and evaluation level
![Page 25: EducationalAssessment](https://reader036.vdocument.in/reader036/viewer/2022062423/568146ff550346895db43c96/html5/thumbnails/25.jpg)
CPR being performed on a mannequin used for training
Inspect Palpate Percuss
Auscultate
OSCE
![Page 26: EducationalAssessment](https://reader036.vdocument.in/reader036/viewer/2022062423/568146ff550346895db43c96/html5/thumbnails/26.jpg)
Objective Structured Clinical Examination
OSCE is a competency-based evaluation aimed at testing the psychomotor and affective domains with an inbuilt system for systematic feedback.
OSCE is a modern type of examination often used in health sciences (e.g. medicine, physical therapy, nursing, pharmacy) to test clinical skill performance and competence such as communication, clinical examination,
![Page 27: EducationalAssessment](https://reader036.vdocument.in/reader036/viewer/2022062423/568146ff550346895db43c96/html5/thumbnails/27.jpg)
Cont…
medical procedures / prescription, exercise prescription, joint mobilization / manipulation techniques, and interpretation of results.
http://www.osceskills.com/
Also known as “Standardized Patient Exams SPEs, described as an acceptable and powerful instrument in clinical performance evaluation… pretended patients in an artificial environment designed to simulate actual clinical condition”
(Borbasi & Koop 1994 as cited in Billing 2005)
![Page 28: EducationalAssessment](https://reader036.vdocument.in/reader036/viewer/2022062423/568146ff550346895db43c96/html5/thumbnails/28.jpg)
Process of OSCE
It normally consists of several short (5-10 minute) stations, in which each is examined on a one-to-one basis with an impartial examiner and either real or simulated patients (actors).
Considered to be an improvement over traditional examination methods because the stations can be standardized enabling fairer peer comparison and complex procedures can be assessed without endangering patients health
![Page 29: EducationalAssessment](https://reader036.vdocument.in/reader036/viewer/2022062423/568146ff550346895db43c96/html5/thumbnails/29.jpg)
Cont…
The other ways an OSCE is made objective is by having a detailed mark scheme and standard set of questions.
For example, a station concerning the demonstration to a simulated patient on how to use a Metered dose inhaler would award points for specific actions which are performed safely and accurately (e.g. candidate explains to patient the need for a seal around the mouthpiece, etc.)
http://www.osceskills.com/
![Page 30: EducationalAssessment](https://reader036.vdocument.in/reader036/viewer/2022062423/568146ff550346895db43c96/html5/thumbnails/30.jpg)
Approaches/Models to OSCE
Multi station OSCE
Scenario based OSCE(sometimes called objstructured clinical assessment (OSCA)
Top to ToeAssessment
OSCE /simulated Assessment with randomallocation
(Rushforth 2007)
Some centers also use viva, or Q/A with these models
OSPE
AKU Med students1st & 2nd year
OSCE
3rd & 4th year
![Page 31: EducationalAssessment](https://reader036.vdocument.in/reader036/viewer/2022062423/568146ff550346895db43c96/html5/thumbnails/31.jpg)
Is OSCE debatable? OSCE stations can never be truly standardized and
objective in the same way as a written exam.
It has been known for different patients / actors to afford more assistance, and for different marking criteria to be applied (literature shows marking criteria as pass/fail, checklist, likert scale, percentages, done/not done)
It provides a formative evaluation for both students and the educational institution.
(Rentschler et al, 2007)
![Page 32: EducationalAssessment](https://reader036.vdocument.in/reader036/viewer/2022062423/568146ff550346895db43c96/html5/thumbnails/32.jpg)
Cont… It is a fast and well-organized evaluation method, allows rapid
feedback to students about their clinical deficits (Billings 2005)
OSCE has been tested and found to be reliable, valid in various settings, provides a controlled clinical situation that is realistic and nonthreatening
# of stations 16 minimum, 37 max (as cited in Rushforth 2007)
Easy in controlled setting but difficult in real world
Literature says that it should be mandatory for nursing students as well
(Rentschler et al, 2007)
![Page 33: EducationalAssessment](https://reader036.vdocument.in/reader036/viewer/2022062423/568146ff550346895db43c96/html5/thumbnails/33.jpg)
Ethical Issues in OSCE
Perceived inequity between students
Lack of honesty and professionalism of peers
http://jme.bmj.com/cgi/content/abstract/32/12/734
![Page 34: EducationalAssessment](https://reader036.vdocument.in/reader036/viewer/2022062423/568146ff550346895db43c96/html5/thumbnails/34.jpg)
Cont… Discuss the OSCE cases with other
students after the OSCE?violation of the honor code.
Why not children as simulated patients?
http://www.oucom.ohiou.edu/AcademicAffairs/predoc/osce/index.htm
![Page 35: EducationalAssessment](https://reader036.vdocument.in/reader036/viewer/2022062423/568146ff550346895db43c96/html5/thumbnails/35.jpg)
Views of students and faculty
Students Overall satisfied Stressful Improve
performance Can practice on
peers as well Encouragement and
motivation
Faculty Overall satisfied Students practice
actively Provide
evaluation to students soon after exam
![Page 36: EducationalAssessment](https://reader036.vdocument.in/reader036/viewer/2022062423/568146ff550346895db43c96/html5/thumbnails/36.jpg)
References Billings,D.M,Halstead,J.A. (2003).Teaching in
nursing: A guide for faculty. 2nd ed.
Bastable,S.B.(2003).Nurse as educator:principles of
teaching and learning for nursing practice. 2nd ed.New York: Jones and Bartlett Publishers.
Hammond,L,D & Bransford, J.(eds) (2005). Preparing
teachers for a changing world:what teachers should learn and be able to do. USA. Jossey Bass. pp 275-326
![Page 37: EducationalAssessment](https://reader036.vdocument.in/reader036/viewer/2022062423/568146ff550346895db43c96/html5/thumbnails/37.jpg)
Cont…http://www.oucom.ohiou.edu/AcademicAffairs/predoc/osce/index.htm
Norman, G. (2002) The long case versus objective structured clinical examinations. BMJ 324: 748-749
Objective Structured Clinical Examination. Retrieved Jan 29, 2009 from http://www.osceskills.com/
Parks,R.,Warren,P.M.,Boyd,K.M., Cameron,H.,Cumming, A., & Jones, J.L. (2006). The Objective Structured Clinical Examination and student collusion: marks do not tell the whole truth. Journal of Medical Ethics,32. 734-738
![Page 38: EducationalAssessment](https://reader036.vdocument.in/reader036/viewer/2022062423/568146ff550346895db43c96/html5/thumbnails/38.jpg)
Cont…
Rentschler,D.D; Eaton,J; Cappiello,J; McNally,S.F; & McWilliam,P.(2007). Evaluation of undergraduate students using objective structured clinical examination. Journal of Nursing Education, 46 (3) 135-139
Rushforth, H.E. (2007). Objective structured clinical examination (OSCE): Review of literature and implications for nursing education. Nurse Education Today ,27, 481-490
Stiggins,R. J.(1999). Assessment, Student Confidence, and School Success. Phi Delta Kappan, 81, (3) Retrieved Jan 23, 2009 from http://www.aac.ab.ca/final 2002.doc
![Page 39: EducationalAssessment](https://reader036.vdocument.in/reader036/viewer/2022062423/568146ff550346895db43c96/html5/thumbnails/39.jpg)
Questions
![Page 40: EducationalAssessment](https://reader036.vdocument.in/reader036/viewer/2022062423/568146ff550346895db43c96/html5/thumbnails/40.jpg)
THANK YOU