1

“Just Do It” Assessment:A Five Step Plan for Busy Faculty

Assessment Bites #4March 1, 2007

Presenter: Mimi Harris Steadman Ed.D.Director of Institutional Assessment

(716) 839-8567 msteadma@daemen.edu

2

Agenda

• Assessment at Daemen• Focused Five Step Assessment Plan• Developing Learning Objectives• Collecting Information• Using Results• Good Assessment

3

Workshop Learning Objectives

Participants will be able to:• Identify different types of assessment at

Daemen College• List the steps of a basic assessment plan• Articulate learning objectives for a course

or program• Describe a variety of direct and indirect

assessment methods

4

Assessment Round Up

What kinds of assessment activities are happening at Daemen?

What information about student learning and institutional effectiveness do we already collect?

5

Assessment at Daemen• Institutional Assessment

– The overarching plan for collecting and reporting information about institutional effectiveness and student learning across the institution.

– Information collected at an institution-wide level, such as enrollment and retention data, and surveys like the NSSE, CIRP and YFCY.

• Core Curriculum Assessment– Assessing student achievement of the core competencies, through direct

methods such as capstone courses, senior projects, and other student work collected via course instructors or e-portfolios, and through indirect methods such as student surveys or focus groups.

• Departmental Assessment– Assessing student learning in majors or programs through direct methods such

as written student work or performances, capstone courses, and success rates on licensure exams or national exams, and through indirect measures such as program reviews, job placement rates, and employer, alumni or student surveys.

• Classroom or Course Assessment– Assessment that determines whether students are learning what we think we are

teaching, and assessment that provides feedback to learners. Direct methods for classroom assessment include assignments, tests, papers, CATs and projects or presentations.

6

5 Step Assessment: ACRRU

BRAINSTORM the basic steps of assessment:

1. A is for…2. C is for…3. R is for…4. R is for…5. U is for…

7

Focused Five Step Assessment Plan (ACRRU)

1. Articulate learning objectives-As a result of this program/course, students should be able to…

(*Offer instruction or other learning experiences*)

2. Collect information/evidence-Using direct and indirect methods

3. Review/analyze data and reflect on findings

4. Report results

5. Use information for decision making and improvement

8

1. Articulate Learning Objectives

• Before we can plan for assessment, we must start by defining learning objectives.

• Learning objectives describe what a student knows, believes, or is able to do.

• Consider the alignment of objectives with your program or institutional mission.

• Learning objectives (or goals) are broader at the institution or general education level and get narrower at the program, department or course level.

• Learning objectives guide course design and instruction.• A few will do. Stagger your assessment and focus on

different objectives each year.

9

Why learning goals and not teaching goals?

The true test of teaching effectiveness lies not in what was taught, but in what was learned. Teaching without learning is just talking. -Tom Angelo and Pat Cross

• In other words, just because I “covered” the material, doesn’t mean my students learned it.

Articulate Learning Objectives

10

Useful learning objectives:

• Describe what we expect our students to know and be able to do

• Use verbs that describe observable action• Describe learner rather than teacher actions• Do not describe course content coverage• Describe an acceptable performance level• Can be assessed by one or more methods• Make sense to colleagues and are clear to

outsidersArticulate Learning Objectives

11

Sample learning objectives

• Program: General Education– Program Goal: Recognize and appreciate artistic and

literary contributions of diverse cultures• Course in Program: Caribbean Literature

– Course Goal: Demonstrate familiarity with themes and genres of classic and contemporary Caribbean literature

(taken from Student Learning Assessment: Options and Resources, MSCHE 1993, p. 14)

Articulate Learning Objectives

12

More sample objectives• Program: History

– Program Level Goal: Communicate orally and in writing about historical topics

• Course in program: Medieval History– Course Goal: Present a cogently-written, critical

analysis of gender and class roles in Medieval England

(taken from Student Learning Assessment: Options and Resources, MSCHE, 1993, p. 14)

Articulate Learning Objectives

13

Developing learning objectivesStart with:• Existing departmental goal statements (including those that just

describe teaching goals rather than learning goals)

• General education goal statements (e.g., critical thinking, lifelong learning)

• Disciplinary organizations

• Professional accrediting organizations

• Assessment plans and reports from other colleges(see http://www2.acs.ncsu.edu/UPA/assmt/resource.htm)

Articulate Learning Objectives

14

Specifying learning objectivesInstructions: With one or two others, role play student and teacher and

generate a response to the following:

“If I’m your student, what do I have to do to convince you that I’m where you want me to be at the end of this lesson, unit or course?”

• _________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Reference: R. M. Diamond (1998). Designing and Assessing Courses and Curriculum. San Francisco: Jossey-Bass, pages 134-135.

Articulate Learning Objectives

15

2. Collect Information/EvidenceUse at least two measures, one direct and one indirect.Direct Measures: direct review of student work on an

exam, a project, or performance (with set criteria for evaluation), licensure or certification exams, etc.

Indirect Measures: student reflections on learning, student, alumni, or employer surveys, focus groups, placement rates, etc.

Data can be new or existing, qualitative or quantitative, collected at the course, program, or institutional level. You don’t need lots of data, just the right data.

If we knew what it was we were doing, it would not be called research, would it? –Albert Einstein

16

Data Collection Resources

See handouts:• Examples of Direct and Indirect Measures

of Student Learning • Quick Tip 6.0

– Data Gathering: Methods• Quick Tip 6.02

– Choosing a Data Gathering Method

Collect Information

17

Do not let the perfect be the enemy of the good.

“To avoid paralysis, a campus is well advised to remember that assessment is an ongoing, flexible, naturally evolving process offering many opportunities to fine-tune or change direction. Meanwhile, even an imperfect process can yield useful results”(Leskes and Wright, 2005, p. 23).

Collect Information

18

3. Review and Reflect on Findings

• Display and discuss findings in a group (ideally in an annual retreat meeting format)

• Start by looking for general trends, then analyze key issues more closely

• Identify areas for action or improvement• Identify additional questions raised and future

focus areas for assessment

The outcome of any serious research can only be to make two questions grown where only one grew before. -Thorstein Veblen

19

4. Report Results

• In a concise format, report on findings, plans for action, plans for future assessment, and resource issues.

• Who are the important audiences for your report, and how will information reach them?

• Are there other venues for sharing assessment findings?

• Don’t forget to celebrate successes. How important it is for us to recognize and celebrate our heroes and

she-roes! -Maya Angelou

20

5. Use Assessment Information for Decision Making & Improvement

“The aim of assessment is primarily to educate and improve student performance, not merely to audit it.”

-Grant Wiggins (1998). Educative Assessment, San Francisco: Jossey-Bass, p. 7.

21

Suskie’s Five Dimensions of Good Assessment

• Used• Cost-Effective• Reasonably accurate and truthful results• Valued• Focus on and flow from important goals

Linda Suskie, Middle States Commission on Higher Education, October 10, 2006 (presented at the IUPUI 2006 Assessment Institute in Indianapolis).

22

Aim for an assessment process that is scholarly and professionally engaging, rather than tedious and time consuming.

Keep your plans lean, focused, and useful so that the assessment process is sustainable and supports, rather than detracts from, the work that matters most.

23

ResourcesMarilee J. Bresciani, Carrie L. Zelna and James A. Anderson. (2004).

Assessing Student Learning and Development. A Handbook for Practitioners. The National Association of Student Personnel Administrators.

Andrea Leskes and Barbara D. Wright (2005). The Art & Science of Assessing General Education Outcomes. Washington, D.C.: The Association of American Colleges and Universities.

Middle States Commission on Higher Education (2003). Student Learning Assessment: Options and Resources. Philadelphia: MSCHE.

Barbara E. Walvoord (2004). Assessment Clear and Simple. A Practical Guide for Institutions, Departments, and General Education. San Francisco: Jossey-Bass.

24

• Instructional Assessment Resources from The University of Texas at Austin, Division of Instructional Innovation and Assessment, (including “Quick Tip” sheets on topics such as rubrics, surveys, qualitative analysis and more). http://www.utexas.edu/academic/diia/assessment/iar/index.php

• Internet Resources for Higher Education Outcomes Assessment from the North Carolina State University Office of University Planning and Analysis. Links to everything related to assessment in higher education, including articles and associations, organized by technique, by discipline, by institution, and more. NCSU hosts the 2007 NC State Undergraduate Assessment Symposium in the Raleigh area April 13 - 15, 2007. http://www2.acs.ncsu.edu/UPA/assmt/resource.htm

• National Assessment Institute sponsored by IUPUI (Indiana University-Purdue University Indianapolis) is a comprehensive assessment conference each fall. http://www.planning.iupui.edu/51.html

• ***COMING SOON: The Daemen College Assessment Web Page***