best practices in college teaching : designing effective rubrics
DESCRIPTION
Best Practices In College Teaching : Designing Effective Rubrics. Debra Dunlap Runshe Instructional Development Specialist University Information Technology Services - Learning Technologies Indiana University – Purdue University Indianapolis. - PowerPoint PPT PresentationTRANSCRIPT
Best Practices In College Teaching: Designing Effective Rubrics
Debra Dunlap Runshe
Instructional Development SpecialistUniversity Information Technology Services - Learning Technologies
Indiana University – Purdue University Indianapolis
Have you ever had a student look at you like this?
Have you ever felt like this?
Do you need a rubric?
If you agree with any of these statements a rubric may be for you…
You are getting carpal tunnel syndrome from writing the same comments on almost every student paper.
You have graded all your papers and worry that the last ones were graded slightly differently from the first ones.
You’ve sometimes been disappointed by whole assignments because all or most of your class turned out to be unaware of academic expectations so basic that you neglected to mention them (e.g., the need for citations or page numbers).
(Stevens & Levi, 2005)
Webinar Objectives
By the end of this webinar, you will be able to:articulate how rubrics can be
useful.describe characteristics of a
rubric.distinguish between analytic
and holistic rubrics.design a rubric.
Where are you now?
Do you use rubrics to assess student work? If so, how?
What do you already know about rubrics?
What are some things you want to learn about using rubrics to assess student work?
Definition of a Rubric
Rubrics are criterion-referenced rules for assessing student performance holistically or analytically (on different dimensions).
A scoring tool that lays out the specific expectations for an assignment, providing a detailed description of what constitutes acceptable or unacceptable levels of performance.
(Stevens & Levi, 2005)
Parts of a Rubric
Task description
A scale
Dimensions of the assignment
Descriptions of performance level
(Stevens & Levi, 2005)
Analytic vs. Holistic Rubrics
Analytic rubric: Common when evaluating independent dimensions or components of student work.
Holistic rubric: Common when the assessed criteria are considered in combination and when quality is judged broadly.
Holistic Rubric
Task Description
Scale Level 1
•Dimension 1 description•Dimension 2 description•Dimension 3 description
ScaleLevel 2
•Dimension 1 description•Dimension 2 description•Dimension 3 description
ScaleLevel 3
•Dimension 1 description•Dimension 2 description•Dimension 3 description
ScaleLevel 4
•Dimension 1 description•Dimension 2 description•Dimension 3 description
ScaleLevel 5
•Dimension 1 description•Dimension 2 description•Dimension 3 description
Holistic Rubric
A•Insightful development and mature style•Cogent analysis of or response to the text•Uses sophisticated sentences effectively
B•Clearly competent•Thoughtful analysis of or response to the text•Less fluent and complex style than 6, but chooses words accurately, varies sentences effectively
C•Satisfactory•Adequate analysis of response to the text•Usually chooses words of sufficient precision, sentences of reasonable variety
D•Unsatisfactory in one or more ways•May analyze or respond to text illogically•Frequently imprecise word choice and little sentence variety
F•Serious weaknesses, of several kinds•Simplistic, inappropriate, or incoherent analysis of or response to text•Inaccurate word choice, monotonous or fragmented sentence structure
Article Review
Analytic Rubric
Scale Level 1
Scale Level 2
Scale Level 3
Scale Level 4
Dimension 1
Dimension 2
Dimension 3
Task Description
Analytic Rubric
Exemplary Competent DevelopingContent Full understanding
of topicGood Understanding of parts of topic
Does not seem to understand topic
Stays on topic Stays on topic 100-95% of the time
Stays on topic 94-75-% of the time
It was hard to tell what the topic was
Preparedness Completely prepared and has obviously rehearsed
Somewhat prepared, but it is clear that rehearsal was lacking
Does not seem at all prepared to present
Eye contact Establishes eye contact with everyone in the room
Sometimes establishes eye contact
Does not look at people during the presentation
Oral Presentation
Constructing a Rubric
A rubric is a protocol for grading based on:
critical componentsscoring scale
So, what is worth grading on (the components)?
What is worth grading on?
Comprehensive literature review
Clarity of rational for chosen research design including importance to field
Understanding of methodology to be used
Includes and addresses potential limitations and implications for practice
Proper references to texts, other resources
Organization, conformity to format
Precision of measurement, quality of data
Clarity of explanations, expression
Strength/tightness of arguments
Grammar and mechanicsWriting styleUse of APA Style
Constructing a Rubric
Four Important Steps
1. Reflection
2. Listing of Objectives
3. Grouping and Labeling
4. Application of Scales
(Stevens & Levi, 2005)
Step 1: Reflection
Eight questions to consider:
1. Why did you create assignment?2. Have you given this or similar assignment before?3. How does the assignment relate to the rest of the
course?4. What skills do students need for successful
completion?5. What exactly is the task assigned?
6. What evidence can students provide to show they have successfully completed the assignment?
7. What does an exemplary product look like?8. What does the worst example of a product look like?
(Stevens & Levi, 2005)
Step 1: Reflection
Objectives
Activities Assessment
Step 2: Listing of Objectives
What are the specific learning objectives for this assignment?
What is the highest level of performance you expect for each learning goal?
Step 3: Grouping and Labeling
Dimensions of an Academic Research Proposal
Research Question
Literature Review
Methodology
Limitations and Significances
Transitions
Grammar and Style
Step 4: Application of Scales
Exemplary, proficient, marginal, unacceptableAdvanced, intermediate high, intermediate, noviceDistinguished, proficient, intermediate, noviceAccomplished, average, developing, beginningOutstanding, very good, good, poor, unsatisfactoryA, B, C, D, FSatisfactory, unsatisfactory
Scoring Scale Example
“Elegance of Argument” component:
5 Original and clearly stated thesis, persuasive, well-organized, imaginative use of source material4 Clearly stated thesis, good use of sources, well
organized3 Facts straight with reasonable explanation of the subject under consideration2 Poorly stated thesis, inadequate survey of available sources, poor organization1 No awareness of argument or complexity
Reliability and Validity
Reliability - Does it get the same results consistently?
Would two experts from the same discipline score student demonstration the same with the rubric?
Validity – Does it measure what it claims to measure?
Would two experts from the same discipline consider what the student is asked to demonstrate an example of what you want to measure?
Let’s Create a Rubric
Think about building a rubric for buying a house.Identify the components that are critical in
completing this task.Write the descriptive
levels for the
components.
Step 1: Reflection
Step 2: Listing of Objectives
Step 3: Grouping and Labeling
Step 4: Application of Scales
DEVELOPING
UNSATISFACTORY
SATISFACTORY
ACCOMPLISHED
PROFICIENT
Advantages of a Clear Rubric
Provides timely feedbackPrepares students to use detailed feedbackEncourages critical thinkingFacilitates communications with othersHelps refine teaching methodsLevels the playing field
(Stevens & Levi, 2005)
Rubrics Help Students to Self-assess
Provide examples of work corresponding to different levels
Provide coaching on components of quality before the whole assignment is undertaken
Have students use the rubric to assess each other’s practice assignments and develop norms of fairness
Have students use the rubric to assess their own work and provide feedback on their self-assessment
Other Considerations
Research other rubrics
Peer evaluation
Revise, revise, revise
eTools for Rubric Construction
RubiStar:
http://rubistar.4teachers.org
iRubric:
http://www.rcampus.com/indexrubric.cfm
AAC&U’s VALUE Rubrics
(Retrieved from http://www.aacu.org/value/rubrics/index_p.cfm?CFID=41453385&CFTOKEN=58615771 August 8, 2012)
VALUE Rubrics
Intellectual and Practical SkillsInquiry and analysisCritical thinking Creative thinking Written communication Oral communication Reading Quantitative literacy Information literacy Teamwork Problem solving
Learning Outcomes for the development of VALUE Rubrics
Personal and Social ResponsibilityCivic knowledge and engagement-
local and global Intercultural knowledge and
competence Ethical reasoning Foundations and skills for lifelong
learning
Integrative and Applied Learning Integrative and applied learning
(Retrieved from http://www.aacu.org/value/abouttherubrics.cfm August 8, 2012)
Summary
During this webinar, we:
discussed how rubrics can be useful.described characteristics of a rubric. distinguished between analytic and holistic rubrics. designed a rubric.
Why use rubrics?
So students look like this …
Thank You for Your Participation!
Debra Dunlap Runshe, Instructional Development SpecialistUniversity Information Technology Services – Learning Technologies
Indiana University-Purdue University IndianapolisInformation Technology and Communications Complex (IT 342H)535 West Michigan Street, Indianapolis, IN 46202
Phone: 317-278-0589 Email: [email protected]
References and Resources
ALTEC at University of Kansas. RubiStar. Web site: http://rubistar.4teachers.org
Carnegie Mellon, Eberly Center for Teaching Excellence. Grading and performance rubrics. Retrieved June 11, 2010 from http://www.cmu.edu/teaching//designteach/teach/rubrics.html
Kansas State University, Office of Assessment. Measures, rubrics, & tools for assessing student learning outcomes. Web site: http://www.k-state.edu/assessment/plans/measures/samples/index.htm
McGonigal, K. (2006, spring). Getting more “teaching” out of “testing” and “grading." Speaking of Teaching, 15, 2. Retrieved June 11, 2010 from http://ctl.stanford.edu/Tomprof/postings/738.html
Moskal, B. M. (2003). Recommendations for developing classroom performance assessments and scoring rubrics [Electronic version]. Practical Assessment, Research & Evaluation, 8(14). Retrieved June 11, 2010 from http://pareonline.net/getvn.asp?v=8&n=14
Moskal, B. M. (2000). Scoring rubrics: what, when and how? [Electronic version]. Practical Assessment, Research & Evaluation, 7(3). Retrieved June 11, 2010 from http://PAREonline.net/getvn.asp?v=7&n=3
References and Resources
Palomba, C.A., & Banta, T.W. (1999). Assessment essentials: Planning, implementing, and improving assessment in higher education. San Francisco: Jossey-Bass.
Quinlan, A. M. (2006). A complete guide to rubrics: Assessment made easy for teachers, K-college. Lanham, MD: Rowman & Littlefield.
Reazon System, Inc. iRubric. Web site: http://www.rcampus.com/indexrubric.cfm
rSmart, a Sakai Commercial Affiliate. Resources for teaching and learning. Web site: http://openedpractices.org/resources
Simkins, M. (1999). Designing great rubrics. Technology & Learning, 20 (1), 23-24, 28-30.
Stevens, D. D. & Levi, A. J. (2005). Introduction to rubrics. Sterling, VA: Stylus.
Suskie, L. (2009). Assessing student learning: A common sense guide. (2nd ed.). San Francisco: Jossey-Bass.
TLT Group. Rubrics. Web site: http://www.tltgroup.org/resources/Rubrics.htm
Walvoord, B.E. (2010). Effective grading: A tool for learning and assessment. San Francisco: Jossey-Bass.
What are rubrics? A rubric is a criterion-referenced scoring tool for assessment linked to learning objectives that is used to assess a student's performance. Rubrics allow for standardized evaluation according to specified criteria, making grading simpler and more transparent.
What benefits do rubrics offer, and to whom?Rubrics benefit both the instructor and the students. They are used both to guide student learning and to assess student learning outcomes.
Creating rubricsSteps in creating rubrics include: 1) articulate the objective(s) of the assignment; 2) identify criteria to be evaluated; 3) determine the levels of performance across the criteria; 4) describe the performance at the various levels for each criterion.
Resources for rubric creationAAC&U’s VALUE rubrics from www.aacu.org/value/metarubrics.cfm Rubistar http://rubistar.4teachers.org iRubric http://www.rcampus.com/indexrubric.cfm
Quick Guide to Rubrics
Rubric Template(Describe here the task or performance that this rubric is designed to evaluate.)
Beginning1
Developing2
Accomplished3
Exemplary4
Score
Stated Objective or Performance
Description of identifiable performance characteristics
reflecting a beginning level of performance.
Description of identifiable performance characteristics reflecting development and
movement toward mastery of performance.
Description of identifiable performance characteristics
reflecting mastery of performance.
Description of identifiable performance characteristics reflecting the highest level of
performance.
Stated Objective or Performance
Description of identifiable performance characteristics
reflecting a beginning level of performance.
Description of identifiable performance characteristics reflecting development and
movement toward mastery of performance.
Description of identifiable performance characteristics
reflecting mastery of performance.
Description of identifiable performance characteristics reflecting the highest level of
performance.
Stated Objective or Performance
Description of identifiable performance characteristics
reflecting a beginning level of performance.
Description of identifiable performance characteristics reflecting development and
movement toward mastery of performance.
Description of identifiable performance characteristics
reflecting mastery of performance.
Description of identifiable performance characteristics reflecting the highest level of
performance.
Stated Objective or Performance
Description of identifiable performance characteristics
reflecting a beginning level of performance.
Description of identifiable performance characteristics reflecting development and
movement toward mastery of performance.
Description of identifiable performance characteristics
reflecting mastery of performance.
Description of identifiable performance characteristics reflecting the highest level of
performance.
Rubric Title (highlight and replace with your title)
(Description of task or performance - highlight and replace with your description.)
Score