Paths to Program Assessment through

Rubrics

Dannelle D. Stevens, Ph.D. Cal Poly, Pomona

May 23 & 24, 2010

Objectives

To learn how to use rubrics for program assessment

Case #1, A sociology dept. listens to its graduates

Case #2, Bottom-up or Top-down Creation

To learn several ways to create and use rubrics for program assessment

To begin to create a rubric for a SLO

Agenda

1. Identifying assumptions about assessment

2. Using rubrics in assessment:

Case #1 – How to use rubrics

3. Creating rubrics

Case #2- Bottom-up, top-down

4. Let’s create a rubric for a program outcome…

What do you need to know?

Basic assumptions about program assessment

Assessment is not new. Yet, grades not enough.

Data, data, seems to be everywhere.

No, not about faculty evaluation.

Yes, about “conversation”, student learning.

Ultimate goals => better programs, student learning

KEY Program Assessment Components

Student Learning

OUTCOMES

STUDENT “WORK”

TIME!A way to

assess quality- RUBRIC

+

+ +

+

3 basic steps

2. GATHER

1. REFLECT/PLAN

3. USE

Case Study #1

Case: An example of SLOs and a rubric to improve student learning.

Context: Sociology Department, Seniors

HANDOUT, Case #1

Product: Oral presentation of Senior Project

Your task

1. Review materials for the case

Context, SLO, Rubric, Scores on a rubric

2. Decide how to use these rubric scores to improve the program.

Questions for program assessment

1. What patterns do you see in the data?

2. What statements could you make based on these data?

3. What SPECIFIC suggestions do you have for the program on how to improve student learning of this outcome? List these and, then, rank order them on a flip chart. Be ready to say why you chose one suggestion over another.

4. What will you tell the graduates?

Case #1- Follow-up

• What did you think about the case?

• Typical?

• Data helpful? Or not?

• Further questions

3 basic steps in program assessment:

Case #1

2. GATHER

1. REFLECT/ PLAN

3. USE

Concern over oral presentations.Identified related Student Learning Outcome.

Gather student work samples.Selected 10 out of 100

Disaggregated the scores.Take notes. Share with others.

Look at the data. Look for patterns.Ask questions.

Make suggestions to department.Follow-up. See what happened.Take notes. Share with others.

K- W- L

• What do you KNOW now about creating rubrics?

• What do you WANT to know?

• What have you LEARNED?

Case #2: Creating rubrics

• Bottom-up ?

• Top- down?

Case #2: Bottom-up Path

• Start with student work

• Read, stack them as exemplary, competent, weak

• LIST short descriptions on stickies of each stack

• GROUP & LABEL the descriptions

• APPLY to rubric grid.

• Check descriptions with SLOs; refine SLOs…

Case #2: Top-down Path

• Start with SLOs. REFLECT.

• LIST many descriptions of how students would demonstrate SLOs at the highest level first.

• GROUP and LABEL descriptions.

• APPLY to rubric grid. Add other levels.

• SCORE sample of student work with rubric. (5-10)

• Refine SLOs or rubric.

Case #2: Discussion

• What is the difference between Bottom-up and Top-Down paths?

• Which makes more sense to you? Why?

• What does this tell you about rubric creation?

Now, let’s begin to create a rubric from a Student Learning Outcome

• Read SLO carefully.

• REFLECT: Imagine student work or performance that would demonstrate this outcome. Think of highest level of performance for now.

• LIST: Many descriptors of student work- evidence that students had achieved this outcome.

• GROUP & LABEL: Group descriptors & label.

• APPLY: Apply.

LISTING

GROUPING & LABELING

APPLYING

The ever-so-generative rubric

What does it generate? Data, useful data

Case #1

Look down the student scores in columns. Choices..

Disaggregate dimensions (e.g, content, style, org.)

Disaggregate scores by student demographics

Can this lead to program improvement. How?

3 basic steps in program assessment

2. GATHER

1. REFLECT/ PLAN

3. USE

Identify potential area of interest/concern.Identify related Student Learning Outcome.

Gather student work, examples of where students should be

demonstrating this outcome.Many decisions here! What kind of evidence?

What class? How many products? Is there a rubric?

Start somewhere. Record the process!

Look at the data.Ask questions.

Use the data to make decisionsto improve the program. Do it!

Then, gather data on that!Take notes. Share with others.

Thank you…

• http://www.csupomona.edu/~academic/programs/programreview/

• www.csupomona.edu/facultycenter

• www.introductiontorubrics.com

Look for rubrics made at University of Alabama