what are outcomes and why should i want to assess them?

1

What Are Outcomes and Why Should I Want to Assess Them?Dr. Jim Wilson, AVP Academic Affairs Wilmington University

Dana Santoro, Data Analyst – Institutional ResearchWilmington University

2

Class of 2011

What Berlin wall? They never “rolled down” a car window They have grown up with bottled water Wal-mart has always been a larger retailer than Sears and has

always employed more workers than GM High definition televisions have always been available China has always been more interested in making money than

in re-education They never saw Johnny Carson live on television Food packaging has always included nutritional labeling

Courtesy of Beloit College Public Affairs

3

Things Have Changed!

Higher Education Research Institute (HERI) UCLA

40yr Trends Presentation

4

Things Have Changed!

Item 1970 1995 Today

Postage Stamp $0.06 $0.32 $0.39

Bread $0.24 $1.40 $1.39

Milk $1.32 $2.59 $4.29

Gas $0.40 $1.35 $2.60

Car $2,500.00 $12,800.00 $27,800.00

Income $6,933.00 $19,717.00 $43,200.00

House $40,000.00 $111,000.00 $264,000.00

5

Education: Costs

Wilmington University Tuition30 credits, Undergraduate

1971

Full year tuition: $1,500

TodayFull year tuition: $8,400 (New Castle, UG)

6

Education: Some Changes

Courtesy of Beloit College Public Affairs

Today Student-centered learning The “Net Generation” Distance Learning, Online databases

1967 Teacher-centered learning Classroom, Books, Library card catalogs Al Gore was only 20 years old

7

Education: Student Learning

A new “paradigm for higher education”(Suskie, 2006.)

Learning CenteredAssessment feedback helps faculty understand

What is and is not working effectively How to improve the curriculum as well as teaching and

learning strategies

8

Wilmington University Mission

9

What is Assessment of Student Learning?

“Good assessment must begin in the classroom and end there.” (Wolvord & Anderson, 1998, p. 149)

10

What is Assessment of Student Learning?

“Systematically gathering, analyzing, and interpreting evidence to determine how well student learning matches our expectations.” (Suskie, 2006, p.3.)

11

What is Assessment of Student Learning?

“Assessment is first and foremost a tool for faculty members to use as they do their very best to teach their students well.” (Middle States, 2003)

12

Assessment Is A Friend Who…

Honestly tells us how we are doing Helps to clarify what we are doing and why Helps us to make informed decisions Improves student learning Helps us to improve Brags about how well we are doing

13

Assessment Could Be a Foe!

Federal requirements Regional Accreditation Legislators Parents and Community groups Students Professional organizations

14

SummaryWhat is Assessment of Student Learning?

Process of gathering & evaluating informationQuantitative and qualitativeFrom multiple sources

To enhance student learning

15

Student Outcomes Assessment Framework

16

Three Assessment Areas

Student Learning

Teaching Effectiveness

Student satisfaction with their academic experience

17

Graduation Competencies

Division / Program Mission

Professional Standards --------

Benchmarks

PROGRAM COMPETENCIES

CECRAMRubrics, Test Embedded Questions, Clinical Evaluations

Data Collection, Analysis, Reporting, Decisions,Actions

Benchmarks Benchmarks

License & Certification Exams

Major Field Exams

• IDEA Results• Faculty Evaluations• GPA Reports

• Enrollment Data •ACT Survey Results

Assessment of Student Satisfaction

Course Work

Assessment of Teaching Effectiveness

Wilmington UniversityACADEMIC AFFAIRS ASSESSMENT PLAN

COLLEGE MISSION

College Outcomes

Assessment of Student Learning Outcomes

18

Program Improvement Cycle

19

Language of Assessment

Direct evidenceDirect evidence of student learning indicates

whether or not a student has a command of specific content

Examples: Homework assignments, research papers, rubrics

CECRAM (Course-Embedded Criterion-Referenced Assessment Measures)

20

Indirect EvidenceIndirect evidence of student learning is

correlational – data exists which indicates that students are probably learning.

Examples: Course grades, time spent on service learning or homework

Language of Assessment

21

Formative assessmentOngoing assessment that is intended to improve an

individual student’s performance.Purpose: to improve course content, provide

feedback to faculty.

Language of Assessment

22

Summative assessmentProvides a true gauge of “outcomes” of student

learning.Data is typically gathered near the end of program

completionExamples: course-embedded project/test/portfolio,

clinical evaluations, standardized exams

Language of Assessment

23

WU Language

“Good enough”The process is accurate and truthful (Suskie, 2004, p. 302)

“Closing the loop”Outcomes data has been analyzed and appropriate

changes have been made.

24

Elements of Assessing Learning

1. Articulated expectation of student learning at various levels.

2. A plan that describes student learning assessment activities.

3. Evidence that student learning assessment information is used.

4. Documented use of student learning assessment as a part of institutional assessment.

25

Background Information

Academic Affairs Assessment Plan adopted in 2003

Plan called for evaluation in 2006

26

MSCHE Outcomes Assessment Guidelines

Assessment should be:UsefulCost-effective, simpleReasonably accurate and

truthfulOrganized, systematized and

sustained.

Suskie, L. (2005)

27

Major Changes Related to Outcomes Assessment

Five Year Data Collection and Implementation Cycle

Revision of Graduation Competencies Representative Sampling Formal Reporting Cycle

Five Year Program Review

28

How to maintain momentum?

A 5 year cycle will be utilized

Year 1 – Implement findings from program

review

Year 2 – Data collection

Year 3 – Reflection, take appropriate action and

collect “other kinds” of data

Year 4 – Data collection

Year 5 – Program Review Report

29

What are Examples?(of closing the loop)

Program: B.S., Finance Weakness: Undergraduate Finance majors were

missing some important finance skills Source: ETS Major Field Exam Action

Adjust the content of FIN 305 Add a new requirement, FIN 306

Competency Disciplined Inquiry Finance program competencies

30

What are Examples?(of closing the loop)

Program: All UG Education programs Weakness: Students were unclear of required skills Source: Portfolio review Actions:

Curriculum changes Provide candidates with exemplars Ensure understanding of required skills

Competency Disciplined inquiry

*Standardized exams 31

Program: M.S., Community Counseling Weakness: Appraisal Techniques

Source: CECRAM, CPCE*, NCE* & surveys

Response: Integration of assessment with diagnosis and treatment planning. Psychopathology course will precede the appraisal course

What are Examples?(of closing the loop)

32

Program: B.S., Nursing Weakness: Student teaching projects - measurable

objectives were not reliably clear Source: CECRAM Actions:

Add content and a teaching video to NUR 425 – Community Health Practitioner

Added teaching content to NUR 330 Revised curriculum and developed a new course:

NUR 323 - Nurse as Teacher

What are Examples?(of closing the loop)

33

What is outcomes assessment of What is outcomes assessment of student learning?student learning?

Why should I support the assessment Why should I support the assessment process?process?

Conclusion

34

References Caffo, Betty, (2003). Striving for Excellence. Faculty Senate

Presentation, Wilmington College. Middle States Commission on Higher Education. (2006).

Characteristics of excellence in higher education. Philadelphia: Author.

Suskie, L. (2004). Assessing student learning: A common sense guide. Bolton, MA: Anker.

Walvoord, B.E., and Anderson, V.J. (1998). Effective grading: A tool for learning and assessment. San Francisco, CA: Jossey-Bass.