2006 fall workshop planning and assessment: a tutorial for new department chairs – a refresher...

2006 Fall Workshop

PLANNING and ASSESSMENT:A TUTORIAL FOR NEW DEPARTMENT CHAIRS –

A REFRESHER COURSE FOR OTHERS

Planning and Assessment Workshop

Session 2

Assessment of Student Learning Outcomes and Departmental Goals

T. Gilmour Reeve

Director of Strategic Planning

Kari Wood

Coordinator, Teaching, Learning, & Technology Center

2:00 – 3:45

November 10, 2006

OUTCOMES ASSESSMENTS OF STUDENTS’ LEARNING

Two boys are walking down the street. The first boy says, “I’ve been really busy this summer. I’ve been teaching my dog to talk.”

His friend responds, “Wow! I can’t wait to have a conversation with your dog.”

The first boy shakes his head. “I said I’ve been teaching him. I didn’t say he learned anything.”

M.J. Allen

Assessing Academic Programs inHigher Education

Planning and Assessment CyclePlanning

(purpose & goals)

Implementation

courses, programs, strategies

Assessment

(measurement & evaluation)

Planning and AssessmentPossible Responses to Assessment:

1. No changes needed; 2. Change Purpose and Goals;3. Change Programs, research, service; 4. Change

Assessments.

Planning

purpose & goals

Implementation

Courses, programs, strategies

Assessment

measurement & evaluation

Levels of Educational Assessment

o InstitutionalSACSCOC – “Institutional Effectiveness”

Addresses all areas and units in delivering all programs and services

o Academic departments and programsDepartmental goals and expected student outcomes for degree programs

o Individual coursesStudent assessment in specific courses & teacher evaluations

Planning and Assessment Workshop – Session 2

There are three basic aspects to these SACSCOC standards:

o The institution DOES planning and assessment;

o The institution USES planning and assessment to improve programs and services;

o The institution DOCUMENTS the planning and assessment activities and the improvements made.

Planning and Assessment Workshop – Session 2

Texas Tech University:

Planning and Assessment

University Policies: OP 10.13 – Strategic Planning and Assessment

“The university, including all academic programs and support operations, is engaged in an ongoing and comprehensive process of planning and assessment. All areas (divisions and colleges) and units (departments, centers, and institutes within areas) must conform to the university policies as specified in this OP.”

Planning and Assessment Workshop – Session 2

Texas Tech University:

Planning and Assessment

University Policies: OP 10.13 (cont.)“Academic programs, including program content, quality,

and assessment, are the primary responsibility of the faculty within the academic disciplines. Learning outcomes assessments of academic programs are included in the departments’ annual assessment reports.”

Planning and Assessment Workshop – Session 2

OP 32.06 – Faculty Responsibility

Specifically, within the first week of the semester, faculty members must provide a course syllabus that includes: a. The course outline;

b. The expected learning outcomes from the course;

c. The methods of assessing those outcomes; and

d. The criteria for grade determination.

Assessment ofStudent Learning

Outcomesin the Courses

Related Terms

(1)TestingAssociated primarily with traditional methods of assessment

(2) EvaluationRequires the interpretation and use of information to make judgments about student learning and/or pedagogy

(3) GradingMay provide insufficient evidence of student learning…Why?

Assessment is an ongoing process of:

1)Establishing clear, measurable outcomes of student learning;

2) Ensuring that students have sufficient opportunities to achieve those outcomes;

3) Systematically gathering, analyzing, and interpreting evidence of student learning; and

4) Using the resulting information to understand and improve the learning process.

Definition

A learning outcome is a specific statement that describes:

(1) knowledge,(2) skills, and(3) attitudes

that students are expected to learn upon successful completion of a course.

Examples

After completing this course, students will be able to:

Art Appreciation Differentiate major works of art

based on the period they represent.

Introduction to Research Methods

Compare and contrast quantitative and qualitative research methodology.

GuidelinesEffective learning outcome

statements:

A = Use concrete, action verbs

C = Are specific to the course

E = Focus on the end, not the means

S = Are student-centered

+A = Are assessable

ACES+A

Action Verbs: Cognitive Domain

Remember

Understand

Apply Analyze Evaluate Create

CountDefineDescribeIdentifyLabelListOutlineQuoteReciteSelect

ComputeConvertDefendDiscussEstimateExplainInferParaphrasePredictSummarize

CalculateClassifyDemonstrateGraphModifyPrepareProduceSolveTranslateUse

AnalyzeArrangeCombineDevelopDiagramDifferentiateDiscriminateIllustrateRelateSeparate

AppraiseAssessCompareContrastCritiqueInterpretJudgeRateSupportTest

ComposeCompileCreateDesignDeviseGenerateIntegratePlanProposeReconstruct

Adapted from Anderson and Krathwohl’s (2001) revised version of Bloom’s taxonomy

Action Verbs: Other Domains

Affective: Argue, convince, debate, express, internalize, justify, support, uphold

Psychomotor: Assemble, build, dissect, dribble, sculpt, repair, sketch, suture

Outcomes are course-specific.

Limit your learning outcomes to 5-10 statements for each course.

These statements will focus on the most important outcomes for the course, not the program or an individual unit.

ACES+A

Focus on the end, not the means.

ACES+A

What should students know or be able to do upon successful completion of the course?

NOT In what activities will students

participate throughout the course?

Outcomes are student-centered.

ACES+A

What should students know or be able to do upon successful completion of the course?

NOT What will instructors do

throughout the course?

Outcomes are assessable.

ACES+A

How can the outcome be measured?

How can we use the information to improve the learning process?

Assessment is an ongoing process of:

1) Establishing clear, measurable outcomes of student learning;

2)Ensuring that students have sufficient opportunities to achieve those outcomes;

3) Systematically gathering, analyzing, and interpreting evidence of student learning; and

4) Using the resulting information to understand and improve the learning process.

Assessment is an ongoing process of:

1) Establishing clear, measurable outcomes of student learning;

2) Ensuring that students have sufficient opportunities to achieve those outcomes;

3)Systematically gathering, analyzing, and interpreting evidence of student learning; and

4) Using the resulting information to understand and improve the learning process.

Best Assessment Practices

1) Assess before, during, and after student learning has occurred.

2) Align assessment methods with the stated learning outcomes.

3) Use a variety of assessment methods.

Framework of Assessment Methods

Traditional Alternative

Selected-Response

Constructed-Response

Product Performance

Process-Focused

Multiple-choiceTrue-FalseMatching

Fill-in-the-blankShort answerLabel a diagram“Show your work”Essay

Research paperLab reportStory/playPoemPortfolioArt exhibitScience projectModelVideo/audiotapePoster presentationPowerpoint

DanceRecital DemonstrationAthletic skillDramatic readingEnactment

DebateTeach a lessonObservationInterviewPresentationDouble-entry journal

http://newton.nap.edu/html/classroom_assessment/ch4_t1.html

Classroom Assessment Technique (CAT)

Short-Writes:

o One Minute Paper: What was the most important thing you learned in class? What question remains unanswered?

o Muddiest Point: What was most confusing about your reading assignment last night?

o Application Cards: Describe one practical application of what we’ve discussed today.

Assessment is an ongoing process of:

1) Establishing clear, measurable outcomes of student learning;

2) Ensuring that students have sufficient opportunities to achieve those outcomes;

3) Systematically gathering, analyzing, and interpreting evidence of student learning; and

4)Using the resulting information to understand and improve the learning process.

Closing the Loop

Are the learning outcomes appropriate?

Is the curriculum aligned with the stated learning outcomes?

Are your instructional methods effective?

Closing the Loop (cont.)

“There is no such thing as effective teaching in the absence of learning. Teaching without learning is just talking.”

-Angelo & Cross (1993)

Planning and Assessment Workshop – Session 2

ASSESSMENT OF STUDENT LEARNING IN THE COURSE

QUESTIONS AND COMMENTS?

Planning and Assessment Workshop – Session 2

ASSESSING STUDENT LEARNING IN THE ACADEMIC PROGRAMS

(PROGRAM-LEVEL ASSESSMENT OF STUDENT LEARNING OUTCOMES)

Planning and Assessment Workshop – Session 2

The term “program-level” refers to learning outcomes and assessment methods related to a degree programProgram-level assessment investigates what the students are learning in their degree programs

Program-level assessment is not the same as the course grades for students.

Planning and Assessment Workshop – Session 2

“Seventy-five percent of students will earn a grade of X [“C” or “B”] or better in Course 1234”

Concern for using course grades as measures in program level assessment:

o A grade is an aggregate of the student’s performances on class assignments (not directly tied to a specific expected outcome)

o Different students will earn the same course grades but will perform differently on course assignments

o Focusing on grades for assessing programs may encourage grade inflation

o Using a minimal course grade of “C” only states what is required for graduation

Planning and Assessment Workshop – Session 2

Program-level learning outcomes and assessment are reflected in the Strategic Plan – on the goals related to undergraduate and graduate education

But the expected outcomes for each degree program do not have to be stated in the STRATEGIC PLANS but are reported in the ANNUAL ASSESSMENT REPORTS

Planning and Assessment Workshop – Session 2

o The first step in writing and assessing program-level learning outcomes is determining the purpose of the degree program

o It is important to engage faculty in this (and all subsequent) steps in the process

Planning and Assessment Workshop – Session 2

Questions to ask to help determine the course purpose include:

o What role does this program play in the department or college?

o What is unique or special about this academic area/degree program?

o What will the students “do” with their degrees?

Planning and Assessment Workshop – Session 2

Once the program’s purpose has been established, specific learning outcomes can be developed

A program-level expected learning outcome is a formal statement of what the program’s faculty expects students to learn in a particular degree program

Planning and Assessment Workshop – Session 2

Expected learning outcomes define:

o What faculty expect students to know at the end of the degree program

ANDo What faculty expect students to

be able to do at the end of the degree program

Planning and Assessment Workshop – Session 2

Once the learning outcomes for a degree have been established, a variety of methods are used to assess if the students’ learning matches those expected outcomes.

Planning and Assessment Workshop – Session 2

Once the learning outcomes for adegree have been established, a

varietyof methods are used to assess if

thestudents’ learning matches those

expected outcomes.

Planning and Assessment Workshop – Session 2

Departments should develop assessment plans that outline

how and when they will be investigating how well both the current students and students

who have graduated have learned/mastered the program-

level learning outcomes

Planning and Assessment Workshop – Session 2

Methods to Assess Current Students’ Learning

o Capstone courseso Review of senior projectso Focus groupso Students’ performance on

comprehensive exams

Planning and Assessment Workshop – Session 2

Methods to Assess Graduated Students’ Learning

o Graduate follow-up surveyso Performance on licensure examso Job placement datao Exit interview/surveys

Planning and Assessment Workshop – Session 2

Strategies for using outcomes assessments of student learning:

o Engage the facultyo Begin with discussions at first faculty

meetings of the yearo Use available resources (TLTC and Office

of Strategic Planning – see websites)o End the year with a faculty review –

make improvements based on assessments .

CLOSE THE LOOP!

Planning and Assessment Workshop – Session 2

See examples in handouts for incorporating outcomes assessments in the Annual Assessment Reports

Also additional information on outcomes assessment is provided in the handouts

Questions and comments?