Rob JohnstoneEducause ELI InstituteJanuary 30, 2012

Making Your Data Work

What we will cover

1. Framing the issue

2. What we learned from the BRIC Initiative, the Aspen Prize, and Completion by Design

3. Making the connection

Building Stronger Systems for SLOs and Program Review – September 2011 2

WHO IS THE RP GROUP?

Putting our work in context

Building Stronger Systems for SLOs and Program Review – September 2011 3

The RP Group

Strengthens the abilities of California community colleges to gather, analyze and act on information in order to strengthen student success

Provides research, evaluation, professional development, and technical assistance services that support evidence-based decision-making and inquiry

Work is defined and conducted by community college practitioners

Building Stronger Systems for SLOs and Program Review – September 2011 4

A RESOURCE FOR INSTITUTIONAL CHANGE

A Culture of Inquiry

An Applied Inquiry Framework for Student Completion – September 2011 5

What is a Culture of Inquiry?

Institutional capacity for supporting open, honest and

collaborative dialog focused on

strengthening the institution

and the outcomes of its students.An Applied Inquiry Framework for Student Completion – September 2011 6

Culture of Inquiry: Why All the Fuss?● Because this ongoing work is challenging

but necessary!● Work needs to marry insight & evidence● Problems are large and recurring● No silver bullet / evident answers● Multiple solutions likely needed● Progress is not linear● Requires cross constituency interaction

An Applied Inquiry Framework for Student Completion – September 2011 7

Culture of Inquiry: Features● Widespread sharing and easy access

to user-friendly information on student outcomes

● Encouraging more people to ask a wider collection of questions and use their evidence and conclusions to enhance decision making

● Shared, reflective and dynamic discussions

An Applied Inquiry Framework for Student Completion – September 2011 8

Culture of Inquiry: More Features● Multiple opportunities to discuss

information within and across constituency groups

● Continuous feedback so adjustments can be made along the way and processes can be adapted

● Culture that values curiosity, questions and robust conversations

An Applied Inquiry Framework for Student Completion – September 2011 9

THE IMPETUS FOR OUR WORK

Why does prioritizing a culture of inquiry make sense?

Building Stronger Systems for SLOs and Program Review – September 2011 10

You’re wasting time and money when your outcomes don’t focus on the right data.

11

Retention Rate

Success Rate

1999 2008

20 year trend for institutional outcomes

What does that tell us about the usefulness of these metrics in setting institutional strategies?

Building Stronger Systems for SLOs and Program Review – September 2011 12

Much of what is needed to support

good decision making isbeyond the

traditional accountability

measures.

13

What is Our College’s Transfer Rate?

Data do not

speak for themselves.

15

The vital role of conversation

● In order to make data useful, ample time and space are needed to discuss and analyze the information and connect it back to the original research question.

● Answers are not always immediately apparent, so skilled facilitation may be needed to dig out the deeper meaning.

● Multiple perspectives and types of information are often needed to make sense of individual data points.

!Building Stronger Systems for SLOs and Program Review – September 2011 16

WHAT CAN WE DO?

What the RP Group has learned

Building Stronger Systems for SLOs and Program Review – September 2011 17

BRIC’s on-campus support helps colleges improve internal practices.

•Technical experts who are community college practitioners

•Focus on discrete practices such as program review

•Opportunities for dialog

Building Stronger Systems for SLOs and Program Review – September 2011 18

Strategy 1:Reform processes

to make themsimpler and

provide more useful

information.

19

age 2: While what colleges developed varied, each system was:

• Meaningful• Sustainable• Consistent

Building Stronger Systems for SLOs and Program Review – September 2011 20

For example …

Comprehensive Assessment Report Exampleshttp://www.rpgroup.org/resources/comprehensive-assessment-

reports-best-practice-examples-around-country

Resource Compendium on Assessmenthttp://www.rpgroup.org/resources/assessment-strategies-and-

models-help-tell-story

Building Stronger Systems for SLOs and Program Review – September 2011 21

Strategy 2: Create space and time to talk so good ideas can

be put into action

22

Get conversations going using inquiry guides.

•Assessing Student Learning Outcomes

•Using an Equity Lens to Assess Student Learning

•Assessing Student Services Outcomes

•Maximizing the Program Review Process

•Assessing Non-credit Student Learning Outcomes

Building Stronger Systems for SLOs and Program Review – September 2011 23

Strategy 3: Make better use of

existing processes;

don’t reinvent the wheel.

24

Strategy 4: Pick your

organizational improvement priorities

and stick to them.

25

Strategy 5: To build lasting changes, ensure you

involve a broad range of

leadership.

26

WHAT THIS MEANS FOR YOU

How can you improve your accountability and success efforts?

Building Stronger Systems for SLOs and Program Review – September 2011 27

Brainstorm & DiscussPick one of the following questions to answer and pair up with someone to share your ideas.

•What process could we simplify at our college?

•How could we create more opportunities for conversations about our student outcomes?

•What is a process we could redesign to make it more useful?

•What should be my college’s top improvement priority?

•How could we build stronger and deeper support for improvement efforts?

Making Your Data Work – November 2011 28

Find Out More

The RP Group Websitewww.rpgroup.org

Rob Johnstone, Senior Research Fellowrjohnstone@rpgroup.org

Bob Gabriner, Leadership Chairgabriner@sfsu.edu

29