assessment made easy maris lown, edd

Assessment Made Easy

Maris Lown, EdD

Assessment answers this question: How do we know students are learning what we think we are

teaching?

Assessment “belongs to the scholarship of teaching.” It is a

study of “situated teaching practices . . . using particular forms

of research and knowledge.”Litterest & Tompkins, (p. 10)

Types of Outcomes

Learning

Describes what we want students to do with the content they are learning.

Success

Identifies whether or not the course or program is effective.

A learning outcome reflects what students can do with the content.

A Success Outcome reflectsa standard of judgment that a discipline, a department or an institution chooses to apply to

students work.

Learning Outcomes• Are measurable• Begin with an action verb• Stress higher-order thinking skills• Are a learner oriented essential ability or skill• Lead to the identification of assessment tools• Identify “what a student is able to do with the

content”• Provide the foundation for success outcomes

Success Outcomes• Are measurable• Begin with a metric• Stress a standard of judgment• Are learner oriented• Reported in terms of numerical scores that

place them in an achievement level based on developed and tested criteria

• Build on learning outcomes

OutcomesLearning

• Describes what we want students to do wth the content.

• Includes a performance indicator

• Give a speech that is designed to convince the audience of a specified point of view.

Success

• Identifies whether or not the course is effective.

• Includes a metric

• 85% of students will achieve a grade of C or better on the course final exam.

10

Comparison Objectives Outcomes

• Yields behaviors which are predictable.

• Primarily concerned with analysis.

• Usually evaluates discrete specific behaviors

• Evaluates behaviors that are most readily assessed and therefore may ignore behaviors that are more important.

• Yields behaviors which are unpredictable.

• Primarily concerned with synthesis.

• Usually evaluates abilities or skills.

• Evaluates outcomes of education which are more difficult to assess and are often ignored.

Assessment: Direct MethodsIntegrated Learning

• Exams• Collaborative projects• Research papers• Videotaped interactions• Journaling• Case studies• Performances• Standardized exams

• Portfolios• Creations• Projects• Internships• Capstone projects• Oral examinations• Concept Maps• Service learning

Assessment: Indirect Methods

Attitudes, Perceptions, Experiences

• Student surveys• Focus groups• Course evaluations• Faculty surveys• CCSSE

• Interviews• Graduate surveys• Employer surveys• Alumni surveys• Student Satisfaction Inventory

“Culture of Assessment”Hierarchy of Assessment of Student Learning

LEVEL 4

LEVEL 3

LEVEL 2

LEVEL 1

What evidence have you gathered

documenting that these changes have Improved Student

Learning?

How have you Changed instruction/the curriculum to Improve

Teaching and Learning based on these Data?

What Data have you Gathered Regarding Achievement of PLOs/CLOs?

What Tools are you using to Assess Achievement of these PLOs?

What are the Program/Course Learning Outcomes?

LEVEL 5

CURRICULUM DEVELOPMENT Institutional Learning Outcomes

Program Learning Outcomes

Course Learning Outcomes

Unit Learning Outcomes

Learning Activities Evaluation Tools

CURRICULUM DEVELOPMENT Institutional Learning Outcomes

Program Learning Outcomes

Course Learning Outcomes

Unit Learning Outcomes

Learning Activities Evaluation Tools

Middle States

• Level 5 requires evidence of changes made due to assessment – departmental minutes.

• We will be required to show a portfolio of assessment projects/artifacts.

• Must be faculty driven.• How can we embed program assessment into

Five Year Program Assessments?• Will we be required to connect learning

outcomes to success outcomes?

Program Review Process

• Connect to PLOs• Can include direct assessment

– Papers, – Exams

• Can include indirect assessment– Student surveys– Exit interviews– Job placement

• Provide samples of program reviews

Institutional Learning Outcomes

• Skills/abilities with which every student graduates.– Critical thinking– Reading– Writing

Program Learning Outcomes

• Connect to institutional learning outcomes.• Includes three or more general education

skills/abilities.• Includes additional discipline skills/abilities.• General education skills/abilities can be

customized to discipline competencies.• Students in programs take ENG 101 and 102.• ETS Proficiency Profile.

Course Learning Outcomes

• Connect to program learning outcomes.• Includes general education skills/abilities

tailored to discipline.• Focuses on discipline skills/abilities.• How many?

Mapping Institutional Learning Outcomes to Program Courses

Program College Wide Learning Outcomes

Program NameCritical Thinking

Communication Teamwork Information Literacy

Biology AS Biology 101Biology 102

English 101Speech 115

English 102Biology 102History 101

English 101

Nursing AAS Nursing 101Nursing 102Nursing 203Nursing 204

English 101English 102 or Speech 115

English 102Nursing 203Nursing 204

English 101

Liberal Studies AA

History 105 English 101Speech 115

English 102 English 101

BiologyProgram Courses

Program Level Student Learning Outcomes

Explain basic biological

principles and defend

scientific perspectives

Apply the scientific

method to hypothetica

l or laboratory problems

Evaluate the

relationship between mankind and the natural world

Utilize critical

thinking skills to

solve biologic

problems.

Communicate scientific material

effectively in oral and written

formats.

Demonstrate information

literacy skills and

technology skills when gathering scientific

information

BIO 111 X X X X X X

BIO 112 X X X X X X

BIO/CHE/PHY electives (2)

X X X X X X

CHE-111 X X X X

CHE-112 X X X X

ENG-101 X X X

ENG-102 X X X

MAT 143 or 171 X X X

MAT 144 or 172 X X X

Name of Program _______BIOLOGY________________________________________ Degree ____AA_______

Mapping Program Learning Outcomes to Program Courses

Mapping PLOs to CLOs, Political Science PLOs CLOs

POLI 101 POLI 102 POLI 220 POLI 221Explore political science methodology

Explain basic political science concepts

Examine the three branches of government and how they check and balance each other

Analyze how states gain and lose power

Investigate the origins of environmental problems

Compare and contrast political ideologies and theories of governance

Interpret political ideologies including conservatism, liberalism, socialism, communism, and fascism

Evaluate the electoral process and how candidates are selected to represent us at the federal level

Compare and contrast the state system and the international system

Analyze the goals of environmental interest groups

Analyze political concepts, theories, and ideologies in spoken and written form

Discuss the strengths and weaknesses of the political process

Compare and contrast the ideological differences of the two major parties

Analyze the contributions of international institutions

Evaluate environmental legislation including the EPS and the CWA

Learning outcomes drive the selection of learning activities and

assessment tools. They lay the foundation for success outcomes.

Mapping CLOs to Learning Activities & Assessment Tools

Course CLO Learning Activity Assessment

NURS I Implement care using the Critical Thinking Model and the Nursing Process

Scaffolded patient case study

Nursing Care Plan

NURS II Assess human needs across the life span

Simulation of a newborn in distress

Newborn Assessment

NURS III Analyze therapeutic communication when interacting with patients

YouTube examples of communication approaches

Video taping patient interaction

NURS IV Implement care focusing on complex pathophysiology

Simulation of a patient in multisystem failure and illness trajectory

Concept Map

NURS IV Collaborate with members of the health care team when delivering care

Management Case Studies

Journal

Collecting Evidence

• Record data• Identify changes made• Collect additional data• Discuss whether changes made a difference• Can be memorialized in department minutes,

annual meeting, or a faculty in-service

Institutional Effectiveness Model

• Strategic Plans– AMP– Technology Master Plan– Facilities Master Plan– Matrix– Non-credit

Institutional Effectiveness Model

• Surveys– Community Needs Assessments– Community College Survey of Student

Engagement– Student Satisfaction Inventory

Institutional Effectiveness Model

• Academic Assessment – Proficiency Profile (e.g.)– Program Assessment– Program Review– Departmental Plans– Data Books– Graduate Surveys– Employer Surveys– Graduation Rates– Transfer Rates

Institutional Effectiveness Model

• Non Credit Assessment– Targeted outcomes assessment

• Administrative Services Assessment– Targeted outcomes assessment– Planning, assessment and budget

• Student Services Assessment– Targeted outcomes assessment

No assertions without evidence Middle States

And now the easy part!

Learning Management Systems

• Applications that ease assessment• Use technology where applicable• Grade books that provide assessment

information

Portfolio Assessment

• Samples of students’ work over time• Uses a rubric• Offers a deep but narrower approach• Gives a richer texture• Can be resource intensive• Can use a sample rather than the entire

population• Can be used for general education assessment• Can be used for prospective employers

Capstone Assessment

• Paralegal Program – Trial Notebook• Incorporates all Program Learning Outcomes• Used at the end of Litigation II• Juried Review• Uses a rubric related to outcomes• Report that includes level 5 and trended data

Assessment of Majors Courses

• Select 2 to 4 courses in the discipline (majors courses, popular courses)

• Select assessment method• Biology

– Assess BIO 111– Assess BIO 112– Assess BIO 105– Assess Bio 106

Embedded Exam Questions

• Department develops questions that relate to student learning outcomes.

• Questions are different for each course but similar in that they assess the same behaviors, e.g.– Analyze an organic compound, analyze an

inorganic compound– Identify normal lung sounds, identify abnormal

lung sounds

Blue Print Assessment ToolsCLO Remembering Understanding Applying Analyzing Evaluating Creating

1 3 23 14 7, 30

2 15 6 20 21

3 7, 27 8, 24

4 13, 25 1 22

5 9 2 26 19

6 10 17, 28 4

7 15 11 18

8 16 5, 29 12

Everything that counts is not countable and everything that is countable does not count.

A. Einstein

What is not countable

• FYE• Trips• Clubs• Speakers• Study abroad

• Athletics• Work study• Service learning

Middle States, 2017

• Preparation, 2011-2014– Assessment process– Budget, planning, assessment– Department plans– Catalog– Syllabi

• Preparation of SSR, 2014-2017

What gets measured get improved!

The central value of learning outcomes assessment is using

evidence and data to make teaching more effective.

Whenever things sound easy,

it turns out there’s one part

you didn’t hear. D. Westlake