rubric development course 20 mac 2019

RUBRIC DEVELOPMENT COURSE

20 MAC 2019

Rohaya TalibSchool of Education FSSH

PhD (Measurement & Evaluation)M.Ed (Measurement & Evaluation)

BBA (Accounting)

At the end of the course, the participants will be able to

• explain the relationship between rubric and Criterion Referenced Assessment (CRA)

• discuss the concept of Performance Assessment (CA)• analyze the anatomy and components of a rubric• evaluate the structure of a valid rubric (parallelism,

homogeneity)• differentiate types of rubric; holistic & analytic• construct an analytic rubric

RUBRIC : Assessment Tool

BRIDGING Teaching

Learning…TOAssessment

Criterion Referenced Assessment (CRA)

Criterion-referenced assessments are designed tomeasure student PERFORMANCE against a fixed setof predetermined criteria or learning standard;concise, written descriptions of what students areexpected to know and be able to do at a specificstage of their education.

www.edglossary.org

ASSESSMENT TOOLs

• Checklist• Rating Scale• Rubric

Criterion referencing is OFTEN defined in terms of proficiency levels.

https://www.edglossary.org/assessment/

Eg Proficiency Level

www.edglossary.org

RUBRICTEST

Performance ?

http://www.ascd.org

Performance-based learning and assessment represent a set of strategies for the acquisition and application of knowledge, skills, and work habits through the performance of tasks that are meaningful and engaging to students.

www.emergingedtech.com

MULTIPLIER EFFECTS OF

PERFORMANCE ASSESSMENT

INDUSTRYCOMMUNITY

www.emergingedtech.com

Gordon Commission on the Future of Assessment in Education, established by the Educational Testing Service in 2011:

Throughout the long history of educational assessment in the United States, it has been seen by policymakers as a means of enforcing accountability for the performance of teachers and schools. But, as long as that remains their primary purpose, assessments will never fully realize their potential to guide and inform teaching and learning. … The problem is that other purposes of assessment, such as providing instructionally relevant feedback to teachers and students, get lost when the sole goal of states is to use them to obtain an estimate of how much students have learned in the course of a year. (Gordon Commission, 2013, p. 7)

The NRC identified several research-based methods for supporting student MASTERY:

National Research Council Board. (2012). Education for life and work – developing transferable knowledge and skills in the 21st century. Washington, DC: National Academies Press.

• Encourage elaboration, questioning, and explanation—for example, prompt students reading a text to think about the author’s intent and/or to explain specific information and arguments as they read, either silently to themselves or to others.

• Engage learners in challenging tasks while also supporting them with guidance, feedback, and encouragement to reflect on their own learning processes.

The NRC identified several research-based methods for supporting student mastery:

National Research Council Board. (2012). Education for life and work – developing transferable knowledge and skills in the 21st century. Washington, DC: National Academies Press.

• Prime student motivation by connecting topics to students’ personal lives and interests, engaging students in problem solving, and drawing attention to the knowledge and skills students are developing and their relevance, rather than grades or scores.

• Use “formative” assessments [evidence-gathering techniques], which continuously monitor students’ progress and provide feedback to educators and students for use in adjusting their teaching and learning strategies. (pp. 9–10)

What is Performance Assessment ?

PA also known as alternative or

authentic assessment, is a form of testing

that requires students to perform a task

rather than select an answer from a ready-

made list.www2.ed.gov

Traditional Assessment Versus Performance Assessment ?

www2.ed.gov

Traditional assessments refer to conventional methods of testing, usually standardized and use pen and paper with multiple-choice, true or false or matching type test items.

Authentic assessments refer to assessments wherein students are asked to perform real-world tasks that demonstrate meaningful application of what they have learned.

Traditional Assessment Versus Performance Assessment ?

Why Performance Assessment is significant?

PA requires students to actively demonstrate what they know…..

PA more valid indicator of students' knowledge and abilities.

www2.ed.gov

What Does The Research Say?

Research suggests that learning how and where information can be applied should be a central part of all curricular areas

Students exhibit greater interest and levels of learning when they are required to organize facts around major concepts and actively construct their own understanding of the concepts in a

rich variety of contexts

PA requires students to structure and apply information, and thereby helps to engage students in Active Learning

www2.ed.gov

Rubrics give structure to observationsMatching your observations of a student's work to the descriptions in the rubric averts (prevents) the rush to

judgment that can occur in classroom evaluation situations.

The rubric describes the performance. The resulting judgment of quality based on a rubric also contains

within it a description of performance that can be used for feedback and teaching.

www2.ed.gov

What Does It Cost?

www2.ed.gov

PA requires a greater expense of

time, planning and thought from

students and teachers.

Richard P. Mills

Commissioner

Vermont Department of Education

Montpelier, VT 05602

Types of Performance

Type of Performance Examples

Processes•Physical skills•Use of equipment•Oral communication•Work habits

•Playing a musical instrument•Preparing a slide for the microscope•Making a speech to the class•Reading aloud•Conversing in a foreign language

Products•Constructed objects•Written essays, themes, reports, term papers•Other academic products that demonstrate understanding of concepts

• Generate scientific analysis report•Laboratory report•Term paper •Written analysis of the effects of the XXX•Model or diagram of a structure (atom, solar, planetary system, etc.)•Concept map

NORMALLY DISTRIBUTED SCORES????

ASSESSMENT

• MEASURE ABILITY,

PERFORMANCE, COMPETENCY, MASTERY

CRITERION-REFERENCED ASSESSMENT

• INTERPRETATION OF SCORE

PERFORMANCE ASSESSMENT

• KNOW AND DO

RUBRIC • SCORING TOOL

1

CHECKLIST2

RATING SCALE3

RUBRIC

?

• Latin word means RED

• An authoritative RULE

• A guide listing specific criteria for grading or scoring academic papers, projects and any

performance assessment

Merriam-webster Dictionary

RUBRIC

A rubric is a scoring tool that explicitly describes the instructor’s performance

expectations for an assignment or piece of work.

https://www.cmu.edu/teaching/assessment/assesslearning/rubrics.html

A rubric is a coherent set of criteria for students' work that includes descriptions of

levels of performance quality on the criteria.

http://www.ascd.org

Effective rubrics have appropriate criteria and well-written descriptions

of performance.

Rubrics help lecturers to: Reduce the time spent grading by allowing lecturers

to refer to a substantive description without writing long comments

Help lecturers more clearly identify strengths and weaknesses across an entire class and adjust their instruction appropriately

Help to ensure consistency across time and across students

Reduce the uncertainty which can accompany grading Discourage complaints about grades


Rubrics help students to:

Understand lecturers’ expectations and standards

Use lecturers’ feedback to improve their performance

Monitor and assess their progress as they work towards clearly indicated goals

Recognize their strengths and weaknesses and direct their efforts accordingly


The main purpose of rubric is to

assess PERFORMANCE

HOLISTIC RUBRICS:Gives students an overall scorefor their paperViews the final product as a set of interrelated tasks contributing to the wholeA score is not assigned for each criterionDoes not provide detailed feedback to the studentIt is efficient and quick.

ANALYTIC RUBRICS:Analytic rubrics allows for educators to provide feedback on each criterionEach part is scored independently to avoid bias.Provides meaningful and specific feedback along multiple dimensionsAre more difficult to construct and is time consuming

VERSUS

http://www.queensu.ca

HOLISTIC RUBRICS:Holistic rubricssingle criteria rubrics (one-dimensional) used to assess participants' overall achievement on an activity or item based on predefined achievement levels;performance descriptions are written in paragraphs and usually in full sentences.

ANALYTIC RUBRICS:Two-dimensional rubrics with levels of achievement as columns and assessment criteria as rows. Allows you to assess participants' achievements based on multiple criteria using a single rubric. You can assign different weights (value) to different criteria and include an overall achievement by totaling the criteria; written in a table form.

VERSUS

http://www.queensu.ca

http://www.queensu.ca/

HRAR

A HR does not list separate levelsof performance for each criterion.Instead, a HR assigns a level ofperformance by assessingperformance across multiplecriteria as a whole.

An AR articulates levels ofperformance for eachcriterion so the lecturer canassess student performanceon each criterion.

SCORE CRITERIA

4(8

0-1

00

%) Research paper demonstrates complete understanding and execution of the assigned objectives.

Thesis statement/argument is clearly stated, complex and original, and the writing does not spend excessive time on any one point of development at the expense of developing other points in the body of the paper. Writing is also error-free, without ambiguity, and reads smoothly, creatively, and with a purpose.

3(7

0-7

9%

) Research paper demonstrates considerable understanding and execution of the assigned objectives. Thesis statement/argument is stated, verges on the complex and original, and the writing shows accuracy and balance in developing body points, but may exhibit occasional weaknesses and lapses in correctness. Writing also has some errors and ambiguities, yet does read clearly and coherently.

2(6

0-6

9%

) Research paper demonstrates some understanding and execution of the assigned objectives. Thesis statement/argument is faintly stated and/or expected and not confident, and the writing is inconsistent in terms of balance in developing body points, and exhibits weaknesses and lapses in correctness. Writing also has many errors and ambiguities, and may read confusingly and incoherently.

1(5

0-5

9%

) Research paper demonstrates limited understanding and execution of the assigned objectives. Thesis statement/argument is simplistic, unoriginal, and/or not present at all, and the writing is unbalanced in developing body points, weak, and incomplete. Writing also has numerous errors and ambiguities, and reads confusingly and incoherently.

Adapted from John Bean, Engaging Ideas, Exhibit 15.4: Holistic Scale for Grading Article Summaries

STANDARDS

CRITERIA ADEQUATE (50-59%) COMPETENT (60-69%) GOOD (70-79%) EXCELLENT (80-100%)

Knowledge of forms, conventions, terminology, and strategies relative to the importance of sources to subject

Demonstrates limited knowledge of forms, conventions, terminology, and strategies relative to importance of sources to subject

Demonstrates some knowledge of forms, conventions, terminology, and strategies relative to importance of sources to subject

Demonstrates considerable knowledge of forms, conventions, terminology, and strategies relative to importance of sources to subject

Demonstrates thorough and insightful knowledge of forms, conventions, terminology, and strategies relative to importance of sources to subject

Critical and creative thinking skills

Uses critical and creative thinking skills with limited effectiveness

Uses critical and creative thinking skills with moderate effectiveness

Uses critical and creative thinking skills with considerable effectiveness

Uses critical and creative thinking skills with a high degree of effectiveness

Communication of information and idea

Communicates information and idea with limited clarity

Communicates information and ideas with some clarity

Communicates information and ideas with considerable clarity

Communicates information and ideas with a high degree of clarity and with confidence

Quality of argument and writing

Argument is simple and unoriginal, and the writing is weak and inconsistent

Argument takes on a fair and expected position, and the writing is moderately clear and coherent

Argument bridges on the complex and original, and the writing is clear and coherent

Argument is complex and original, and the writing is strong, fluid, and creatively coherent

Spelling and grammar Several errors in spelling and grammar

A few errors in spelling and grammar Some errors in spelling and grammar

No errors in spelling and grammar

Adapted from Centre for Teaching Excellence, Appendix B: Sample Analytic Rubric (“Rubrics: Useful Assessment Tools.

An Analytic Scoring Rubric

Carriveau R.S. (2016), Connecting the dots: developing student learning outcomes and outcome based assessments (2nd ed, Stylus Publications, Sterling, VA)

• An analytic rubric is an assessment instrument for scoring a constructed response. The response can be a written response, a performance, or a product.

• The response can be from an individual or a group

• An analytic rubric includes a score scale of 1 to 3 points or up to 1 to 5 points with criteria and descriptors for assigning the points

Anatomy of An Analytic Rubric (CDPL)

• the aspects of performance (e.g., argument, evidence, clarity) that will be assessed

• the characteristics associated with each dimension (e.g., argument is demonstrable and original, evidence is diverse and compelling)

• a rating scale that identifies students’ level of mastery within each criterion


Criteria

Descriptors

Performance

Levels

Construction of Rubric Structure

The two basic principles in rubric construction are parallelism in structure and homogeneity in content

Parallelism means the basic grammatical structure of the descriptions are similar from row cell (column 1) to row cell (column 2…)

Homogeneity means the content is similar for row cell (column 1) to row cell (column 2…) so that you are measuring the same content.

teachingcommons.unt.edu

DES

CR

IPTO

RS

Allen, M.J. (2004).Assessing academicprograms in highereducation. Bolton,MA: Anker.

Criteria/ Outcomes

4 3 2 1

Position Statement[PS]: CLARITY30%

PS is clearly stated and is consistently maintained throughout the paper.

PS is clearly statedbut shifts slightly throughout the paper.

PS is not clearly stated and shifts throughout the paper.

An attempt at a position statement is evident, but it cannot be clearly determined.

Structure of the argument (SoA) : CLEARTRANSITION50%

Structure of the argument is well developed with clear in alltransitions.

Structure of the argument is moderately well developed, and most transitions are clear

Structure of the argument is moderately well developed, and many transitions are weak and unclear.

Structure of the argument is weak, and most transitions are weak and unclear.

Rubric for Position Paper [an essay that presents an arguable opinion about an issue]

Criteria/ Outcomes

4 3 2 1



PS is clearly statedbut shifts slightly throughout the paper.



Rubric for Position Paper: is an essay that presents an arguable opinion about an issue

Criteria: the aspects of

performance

Descriptors: the characteristics

associated with each dimension

Performance levels: a rating scale that identifies students’ level of mastery within each criterion

1 2 3

1 2 222

3 3

Criteria/ Outcomes

4 3 2 1



PS is clearly stated but shifts slightly throughout the paper.



Rubric for Position Paper

The Value of Consistency in Terms of Structure And Content

• The primary focus of assessment is to determine what the student knows and can do so that instructional and evaluation decisions can be made.

• To make valid decisions about a student requires well-constructed assessment instruments (rubrics) that consistently produce valid scores.

• Validity means that the scores can be used and interpreted as they were intended to be used and interpreted.


The Value of Consistency in Terms of Structure And Content

• Parallel structure and homogeneous content help ensure rubric score validity because they make it easier for the rater or raters to more objectively score the student’s work.

• The rater needs to be able to see clearly the progression from a high to a low quality student response.

• Any inconsistency or confusing language would reduce the ability of the rater to make valid decisions.


Criteria/ Outcomes

4 3 2 1



PS is clearly stated but shifts slightly throughout the paper.



Rubric for Position Paper

Score

Copyright 2018, Jon Mueller. Professor of Psychology, North Central College, Naperville, IL.

How Many Levels of Performance Should I Include in my Rubric?

It is not easy to write good descriptors for each level and eachcriterion. So, when you first construct and use a rubric you mightnot include descriptors. That is okay. You might just include thecriteria and some type of labels for the levels of performance.

Once you have used the rubric and identified student work thatfits into each level it will become easier to articulate what youmean by "good" or "excellent." Thus, you might add or expandupon descriptors the next time you use the rubric.

http://www.noctrl.edu/


A rubric can have as few as two levels of performance (e.g., a checklist) or as many as ... well, as many as you decide is appropriate.

There is no "right" answer as to how many levels of performance there should be for a criterion in an analytic rubric; that will depend on the nature of the task assigned, the criteria being evaluated, the students involved and your purposes and preferences.



Level of Performance


Students better understandwhat good (or bad)performance on a task looks likeif levels of performance areidentified, particularly ifdescriptors for each level areincluded




Also, it is NOT TRUE that there must be an EVEN number or an ODD number of levels. Again, that will depend on the situation.



4 Level of Performance in the Rubric

POOR GOOD EXCELLENT SUPERIOR

BEGINNING BASIC PROFICIENT ADVANCED

SELDOM SOMETIMES USUALLY OFTEN

UNACCEPTABLE MARGINAL PROFICIENT DISTINGUISHED

BEGINNING DEVELOPING COMPETENT EXEMPLARY

NEEDS IMPROVEMENT

SATISFACTORY GOOD ACCOMPLISHED

UNACCEPTABLE MARGINAL PROFICIENT DISTINGUISHED

NEEDS IMPROVEMENT

SEVELOPING SUFFICIENT ABOVE AVERAGE

resources.depaul.edu

Center for Teaching and Learning25 E. Jackson Blvd.Chicago, IL 60604(312) 362-8000


NEEDS IMPROVEMENT MEETS EXPECTATION EXCEED EXPECTATION

BASIC PROFIENT ADVANCED

POOR MINIMAL SUFFICIENT ABOVE AVERAGE EXCELLENT

NOVICE INTERMEDIATE PROFICIENT DISTINGUISHED MASTER

UNACCEPTA

BLE

EMERGING MINIMALLY

ACCEPTABLE

ACCEPTABLE ACCOMPLISHED


resources.depaul.edu

Center for Teaching and Learning25 E. Jackson Blvd.Chicago, IL 60604(312) 362-8000

SUMMARY

Rubrics are NOT used to compare the performance of students, but TO COMPARE A

SINGLE STUDENT AGAINST THE SET OF CRITERIA.




COURSE INFO

PROJECT-BASED ASSESSMENT

PROJECT OUTLINE PRESENTATION (FORMATIVE) 5% CS/ KW

PROJECT REPORT 20% APTEAMWORKING 5% TS



COURSE INFO

PROJECT-BASED ASSESSMENT

PROJECT OUTLINE PRESENTATION (FORMATIVE) 5% CS

COMMUNICATION

SKILL RUBRIC


5. Select levels of perfomance

r

1. Select a performance assessment for a analytical rubric development

2. Draw a table (start small ok !)

3. Decide the outcomes / criteria

6. Construct the descriptions / descriptors (Parallelism & Homogeneity apply)

4. Determine the weightage for each criterion

Try to develop a rubric on

COMMUNICATION SKILL (CS)

OBE SYSTEMC1,C2,C3…..

Tools To Develop Rubric

• Rubric Generator

• iRubric

• Rubistar

rubric development course 20 mac 2019

Documents