introduction of grading rubrics to egyptian pharmacy students’ oral assessments
TRANSCRIPT
Introduction of Grading Rubrics to Egyptian Pharmacy Students’ Oral
Assessments
Dalia A. Hamdy
BPSc, MSc, PhD, RP(ACP), MRSCAssistant Professor
Faculty of Pharmacy-Alexandria UniversityEgyptian Young Academy of Sciences (EYAS) Member
Oral Rubrics: Outline
• Oral Exams: Introduction• Oral Rubric: Pharm-AlexU Experience– Methodology– Results– Conclusions– Limitations– Updates
Oral Exam Rubrics-DH-April 2016 2
3
Oral Exams: Introduction• Teachers (Facilitators)help students develop and attain skills, knowledge and attitudes.
• Assessorscertify their competency to join a discipline, a community or a profession
Oral Exam Rubrics-DH-April 2016
4
Oral Exams: Introduction• Higher Education Assessment methodsShould be
• aligned with both learning outcomes and activities• utilized for educational improvement
Oral Exam Rubrics-DH-April 2016
which students
learn best
under what
condition
5
Oral Exams: Introduction• In designing any exam for grading students
work
Oral Exam Rubrics-DH-April 2016
What is being
Assessed?
6
Oral Exams: Introduction• In designing any exam for grading students
work we need to
-Identify and clarify assessment standards and criteria
-share such criteria with the student before-hand
Oral Exam Rubrics-DH-April 2016
7
Oral Exams: IntroductionThe Pharmacy program curricula in the Egyptian Universities have three exams for students’ assessment
1. Written exams (40-50%)2. Practical exams (20-30%) 3. Oral exams (10-20%)
Oral Exam Rubrics-DH-April 2016
8
Oral Exams: IntroductionOral exams • Assessment in which students responds verbally to the
assessment tasks
• Enabling students’ assessment on all five cognitive domains of Bloom’s taxonomy
• Limitation: concerns regarding subjectivity of oral exams, specially with the involvement of a large number of faculty staff members to handle the large number of students present in the Egyptian Universities.
Oral Exam Rubrics-DH-April 2016
9
Oral Exams: IntroductionOral exams
Oral Exam Rubrics-DH-April 2016
10
Oral Exams: IntroductionOral exams• Development & use of appropriate rubrics can
be used to – decrease subjectivity – Ensure consistency across time and across graders
• Rubrics are scoring/evaluation tools that describe the assessment performance expectations
Oral Exam Rubrics-DH-April 2016
Oral Exam Rubrics-DH-April 2016 11
Oral Rubric: Pharm-AlexU Experience
Methodology:• The oral exam targeted areas were defined as
testing students’ 1. knowledge2. communication skills, 3. problem solving 4. critical thinking ability
Oral Exam Rubrics-DH-April 2016 12
Oral Rubric: Pharm-AlexU Experience
Methodology:• Study took Place in January-June 2013• A rubric was developed
Oral Exam Rubrics-DH-April 2016 13
Oral Rubric: Pharm-AlexU Experience
Methodology:• A rubric was developed
Oral Exam Rubrics-DH-April 2016 14
Oral Rubric: Pharm-AlexU Experience
Methodology:• A rubric was developed
• An oral question or assessment was assigned to test each item in the rubric.
• A sample of 42 pharmacy undergraduate students, year 1 clinical program, were assessed
Oral Exam Rubrics-DH-April 2016 15
Oral Rubric: Pharm-AlexU Experience
Methodology:
Assessor 1 Assessor 23 years of experience 25 years of experience
Applied the rubric -Did not apply the rubric-Asked the students to go over the
questions in a provided sheet + other questions
Tested 3-4 students at a time Tested 1 student at a time
Oral Exam Rubrics-DH-April 2016 16
Oral Rubric: Pharm-AlexU Experience
Methodology:
-Grades were out of 15 by each assessor
-The mean of both grades would be used as their final grade
Oral Exam Rubrics-DH-April 2016 17
Oral Rubric: Pharm-AlexU Experience
Results:
Assessor 1Assessor 2
5
3
Total Assessment time (h)
Assessor 1Assessor 2
7
5
Average time per student (min)
-Assessor 1 required computer assistance-A developed excel sheet was used to convert the rubric grades of each criterion with its weight to real grades for each student.
Oral Exam Rubrics-DH-April 2016 18
Oral Rubric: Pharm-AlexU Experience
Results:
0 ≤2 ≥3 and ≤6
15%
44% 41%
The difference between the grades given for the same student
Oral Exam Rubrics-DH-April 2016 19
Oral Rubric: Pharm-AlexU Experience
Results:
8 9 10 11 12 13 14 15-1
1
3
5
7
9
11
13
15
Assesor 1 Assesor 2
Grade
Num
ber o
f Stu
dent
sA histogram representing the distribution of students grades by different assessors
Oral Exam Rubrics-DH-April 2016 20
Oral Rubric: Pharm-AlexU Experience
Conclusion:-Oral assessments are an essential component of pharmacy students’ education
-They provide the students with an opportunity to -use higher-order thinking skills -communicate information to patients &
healthcare professionals
Oral Exam Rubrics-DH-April 2016 21
Oral Rubric: Pharm-AlexU Experience
Conclusion:-Points of strength:
1. The use of the mixed modules; students are provided with a question sheet at the beginning of the exam that they need to answer in writing before they enter their oral assessment
2. The assessors involved in the exam are PhD holders at the rank of at least Assistant professors
Oral Exam Rubrics-DH-April 2016 22
Oral Rubric: Pharm-AlexU Experience
Conclusion:-Points of strength:
3. each student is assessed by one assessor, however, in small groups, as in clinical pharmacy programs, the student is assessed by more than one assessor.
Oral Exam Rubrics-DH-April 2016 23
Oral Rubric: Pharm-AlexU Experience
Conclusion:-Suggestion:• Implementation of two additional steps to increase the
validity of the oral exam and decrease subjectivity and inter-rater variability among assessors.
First step: introduction of a grading rubric Second step: train the examiners on rubric use specially newly hired ones who lack experience
Oral Exam Rubrics-DH-April 2016 24
Oral Rubric: Pharm-AlexU Experience
Conclusion:-Advantage:1. Better student assessment to include different
intellectual skills & better identification of learning gaps
2. Decrease subjectivity and increase caliber of the newly hired staff.
3. Better student satisfaction and less stress due to clarity of assessment criteria
Oral Exam Rubrics-DH-April 2016 25
Oral Rubric: Pharm-AlexU Experience
Conclusion:-Limitation:
1. Time (experience & training)
2. Computer availability (updates)
Oral Exam Rubrics-DH-April 2016 26
Updatesimplemented
in June 2014
&2015
Thank You!
27Oral Exam Rubrics-DH-April 2016