“Closing the Loop”: Interpreting and Reporting

Results

@CampusLabsCo #labgab Like us on Facebook!www.campuslabs.com/blog

Assessment isany effort to gather, analyze, and

evidence which describes institutional, departmental, or agency effectiveness.

INTERPRET

The Assessment Cycle

Share and use results to inform

decisions and practices; make improvements

to student learning

Review assessment

results

Determine assessment method(s)

Develop or revise divisional, departmental, or program goals, objectives, and outcomes

Identify or develop and administer

measure(s); collect data

Provide experiences (programs, services, activities, classes)

1

2

3

6

5

4

Give yourself a head start!

Consider what kind of data you need

Quick snapshots

(KPI’s)

Stories

(Rubrics/Coded data)

Numbers

(Export a saved view)

Narratives

(Export Open Responses)

Internal comparisons

(Comparative reports)

External comparisons

(National benchmarks)

Developing questions• Be clear.

• Avoid biased, loaded or leading questions.

• Avoid making significant memory demands.

• Keep the question short.

• Don’t make assumptions.

• Define terms and concepts.

• Consider whether the respondents will feel comfortable answering honestly.

• Let participants admit they don’t know or can’t remember.

• Multiple questions (double-barreled)

• Socially desirable responses

Note the reason for each data point

• Bubble next to question

• Compare against purpose to identify gaps

• Look for overlap

• Eliminate “nice to know”

• Help with ordering

• Retain for analysis step

Comparison reports

Closing the loop requires…

• Context

• Ability to analyze data (or access to someone who does)

• Time

• Ability to translate/explain

CLOSING THE LOOP

Analyze data

Make recommendations

Share results

Document action

Step 1

Step 2

Step 3

Step 4

Writing recommendations

• Explain your rationale

• Reference, don’t repeat the findings

• Avoid introducing new material

• Include context

• Be specific

• Be realistic

• Give them options: Include many levels of recommendations (short-term and long-term, low budget and high budget)

• Consider what you can do

Improve it!

As a result of the survey, in the future the hazing awareness program needs to cater to the specific challenges and needs of the different councils. Based on the survey, less than 200 student responded that the hazing awareness ads were moderately beneficial or higher. The cost for the ads was $2,450 spending approximately $2.42 per students who actually responded. As a result, ads should continue. A round table discussion of the results with the students after all data has been gathered will be scheduled.

Improve it!

Tier 1 intramural groups are amongst the few organizations participating in off-season training and would therefore like to see more opportunities for strength and conditioning. The large number of responses reflecting dissatisfaction likely stems from a lack of an organized strength and conditioning program, something we may explore further.

Improve it!

These results illustrate the range of critical transferable skills that are typically available for a student to develop through participation in a student employment experience. These results will be shared with students to help them take full advantage of their work experiences to enhance their professional future. They also establish a baseline so the Career Center can measure the impact of future efforts to assist students to further develop the learning potential of these experiences.

For your consideration:

• Length and depth of your recommendations will depend on the situation

• Use others for brainstorming

• Ask for feedback on your first draft

• Practice makes perfect!

CLOSING THE LOOP

Analyze data

Make recommendations

Share results

Document action

Step 1

Step 2

Step 3

Step 4

Step 3: Share results

Why do we do this?

Recognition/validation

Responsibility/Accountability

Increases participation

Reinforces importance

of assessment

Promotes institutional

transparency

Helps others be more educated

Combats survey fatigue

Reduces chance of redundant

data collection

Sharing results begins with…

•Clear objective/purpose to your project

•Idea of audience (who cares about the results)

•Appropriate analysis of data

•Effective reporting/presentation of results

•Role modeling…you use the data, others will follow!

Policymakers are more likely to read information if:• Information is in short bulleted paragraphs, not

large blocks of type

• Charts or graphs are used to illustrate key points

• It’s provided in print rather than electronically (67%), but 27% indicate reading electronic materials more often.

• Recommendations and implications are presented

Matching Format to AudiencesWITH STUDENTS

WITH INTERNAL CAMPUS PARTNERS

WITH EXTERNALCONSTITUENTS

WITH STAFF

• Email invitations

• PR campaign (flyers,

newspaper, TVs)

• Student gov’t meetings

• Roadshows

• Brief emails

• Newsletters

• Retreat

• Full Reports

• Cabinet meetings

• Elevator speech

• Exec. Summary

• Annual Reports

• Presentations

• Website

• Press releases

Executive Summary

Why?

Who?

What?

So what?

Annual Report

http://www.albany.edu/studentaffairs/assessment/docs/Briefing%20Book%2008-09/index.html

PR or Advertising Campaign

St. Louis University Infographic on Student Involvement

Annual Report

Annual Report

Websites

Websites

Sharing Internally

Presentation

Symposiums

Did you know email

Newsletters

Flyers

Flyers

Performance Indicators

Survey Finish Page

Survey Invitation

Poster Proposal Tree (and Repository)

http://studentaffairs.duke.edu/ra/assessment/2nd-annual-assessment-and-evaluation-poster-fair

Other ideas…

Student facing website (perhaps even interactive)

Knowledge-sharing day (invite students too!) Lesley University

Videos UTEP

Tweets or Facebook Posts

Reporting Advice

1. Seek out example reports/methods for sharing

2. Consider adopting a consistent style for reports/sharing

3. Develop an electronic template for reports/sharing

4. Adopt an “Assessment Spotlight” at staff meetings

5. Figure out how (not if!) to share results with students

Overall goal:

Share your results in a variety of ways with a variety of audiences in ways that are meaningful

Step 4: Document Action Taken

Answer the questions:

What did we do differently?

What did we continue to do the same?

How were students (or others) impacted?

Make the most of your data.

See the impact on students.

CLOSE THE LOOP.

Jenna Ralicki

Consultant, Campus Success

jralicki@campuslabs.com

716.270.0000

QUESTIONS?