Creating SLO Assessment Plans

Presented by ORIE Team

Summer 2013

Academy for Planning, Assessment, and Research

Table of Contents

I. Defining Student Learning OutcomesII. Creating PLO or CLO Statements

A. Bloom’s TaxonomyB. Habits of MindIII. Selecting Measures A. Direct vs. Indirect MeasuresB. Program Level Measures vs. Course Level MeasuresIV. Selecting Evaluation MethodsA. Score Weighting Guide B. RubricsVI. Setting Performance Criteria for Individual StudentsVII. Specifying Target Outcome for Group Success in meeting the SLO

Defining Student Learning Outcomes

The student learning outcome is a statement that explains what the student is learning, including the accumulated and demonstrated knowledge, skills, abilities, behaviors, and habits of mind, as a result of actively participating in the course or program of study.

Program Learning Outcomes answer the question:

What will students know or be able to do when they exit the program?

Course Learning Outcomes answer the question:

What will students know or be able to do when they complete the course?

Creating Student Learning Outcome

StatementsStudent Learning Outcome StatementsShould align with the purpose of the program or the course;Should be specific and measurable, using action verbs.

Bloom’s Taxonomy--Three Learning Domains:Cognitive

Affective

Psychomotor

Bloom’s Taxonomy

Knowledge: Recall data or information

cite, count, define, describe, draw, identify, know, label, list, match, name, outline, point, quote, read, recall, recite, record, recognize, repeat, reproduce, select, state, tabulate, tell, trace, underline.

SLO Examples:

Students will be able to

Describe history, purpose, and scope of physical therapy

Identify the social, political, economic and cultural influences and differences that affect the development process of the individual.

Bloom’s Taxonomy

Comprehension: Understand the meaning, translation, interpolation, and interpretation of instructions and problems. State a problem in one's own words.

comprehend, compute, convert, defend, differentiate, discuss, distinguish, estimate, explain, express, extrapolate, extend, generalize, give examples, infer, interpret, paraphrase, predict, report, restate, review, rewrite, summarize, translate.

Examples of SLO:

Students will be able to

Distinguish important aspects of the western moral theories, from the virtue theory of ancient philosophers to the modern theories of act ethics.

Explain georeferencing of photos or images to maps.

Summarize the principles of magnetism.

Bloom’s Taxonomy

Application: Use a concept in a new situation or applies what was learned in the classroom into novel situations in the work place.apply, calculate, change, compute, construct, demonstrate, determine, discover, dramatize, employ, examine, illustrate, interpret, locate, manipulate, modify, operate, predict, prepare, produce, relate, show, solve, use.

SLO Examples

Students will be able to

Implement plan of care for patients and families within the legal, ethical, and regulatory parameters.

Apply appropriate organizational skills for selected types of speeches

Conduct basic laboratory experiments involving classical mechanics.

Bloom’s Taxonomy

Analysis: Separates material or concepts into component parts so that its organizational structure may be understood. Distinguishes between facts and inferences.

analyze, appraise, break down, compare, contrast, diagram, deconstruct, differentiate, discriminate, distinguish, identify, illustrate, infer, outline, relate, select, separate.

SLO Examples:

Students will be able to

Analyze and designelectrical and electroniccircuits and systems, using Knowledge of mathematics and basic sciences.

Analyze self-practice in relation to the roles of the professional nurse.

Bloom’s Taxonomy

Synthesis: Builds a structure or pattern from diverse elements. Put parts together to form a whole, with emphasis on creating a new meaning or structure.

categorize, combine, compile, compose, create, devise, design, explain, generate, integrate, modify, organize, plan, rearrange, reconstruct, relate, reorganize, revise, rewrite, summarize, tell, write

SLO Examples:

Students will be able to

Integrate the pathophysiological assessment findings to formulate a field impression.

Coordinate human information and material resources in providing care for patients and their families.

Bloom’s Taxonomy

Evaluation: Make judgments about the value of ideas or materials.

appraise, compare, conclude, contrast, criticize, critique, defend, describe, discriminate, evaluate, explain, interpret, justify, relate, summarize, support

SLO Examples:

Students will be able to

Use critical thinking and a systematic problem-solving process for providing comprehensive care.

Analyze and evaluate economic events that apply to the preparation of financial statements.

Habits of Mind

Arthur Costa and Bena Kallick (2000)’s Habits of mind: Persisting, thinking flexibly, and striving for accuracyExamples of Habits of Mind: Develop and use

effective time management skills

Follow directions correctly

Appreciate negatives and failures as opportunities to learn and discover choices

SLO Examples:

Students will demonstrate time management skills, adhering to all deadlines for assignments, tests, and projects.

Students will demonstrate effective therapeutic patient communication/relationship.

Selecting Measures

A. Direct vs. Indirect MeasuresTo measure student learning outcomes, use the direct measures primarily, but also use indirect measures for triangulation of data Direct Measures: Refer to the type of data that can be used to directly measure students’ knowledge and skills

Examples: Pre-test, post-test, comprehensive subject matter exam, licensure exam, portfolio, thesis, writing assignment, internship evaluation, capstone assignment, and performance demonstration (recital, art exhibit, or science project)

Selecting Measures

Indirect Measures: Refer to the type of data that can be used to infer student learning or achievement

Examples: Surveys, Interviews, Focus Group Studies, Document Analyses, or Audit Measures (course grades, retention/graduation/job placement rates)

Selecting Measures

B. Program Level Measures vs. Course Level Measures

CLO Measures:

Exams, TestsRubric-Based Assignments: ProjectsEssaysPortfoliosPresentationsPerformance Demo.

PLO Measures:

Exit exam (standardized, licensure/professional exams)Thesis, DissertationCapstone Course Paper or ProjectPortfolios

Selecting Measures

C. How to Select Measures?

Measuring Knowledge, understanding

Tests, Exams, Essays, Portfolios, Presentations

Measuring Skills Performance Demonstrations, Projects, Presentations, Internship Evaluation

Measuring Habits of Mind

Surveys, Interviews, Behavior Observations, Focus Groups, Document Audit

Examples of Measures Aligned with SLOs

SLOs Measures (Data Related to Student Work /Performance /Opinion)

Students will be able to describe history, purpose, and scope of physical therapy(Knowledge)

Exam on the history and purpose of Physical TherapyEssay describing the scope of physical therapy

Students will be able to distinguish important aspects of the western moral theories, from the virtue theory of ancient philosophers to the modern theories of act ethics. (Understanding)

Exam or Essay Questions on the Exam summarizing the principles of ethical theoriesEssay comparing the similarities and differences between the ancient and modern theories

Examples of Measures Aligned with SLOs

SLOs Measures (Data Related to Student Work /Performance /Opinion)

Students will be able to apply appropriate organizational skills for selected types of speeches. (Skills)

Demonstration of public speakingSpeech in writing

Students will demonstrate time management skills, adhering to all deadlines for assignments, tests, and projects. (Habit of Mind)

Grade Book AuditSurvey on student time management skills

Selecting Evaluation Methods Aligned with

MeasuresMeasures (Data Related to Student Work /Performance /Opinion)

Evaluation Methods (How do instructors assess students’ work or performance?)

Exam on the history and purpose of Physical TherapyEssay describing the scope of physical therapy

Score Weighting Guide with scores assigned to each exam questionGrading Rubric specifying levels of quality in identified areas

Essay Questions on the Exam summarizing the principles of ethical theoriesEssay comparing the similarities and differences between the ancient and modern theories

Grading Rubric specifying levels of quality in identified areas

Grading Rubric specifying levels of quality in identified areas

Selecting Evaluation Methods Aligned with

Measures

Measures (Data Related to Student Work /Performance /Opinion)

Evaluation Methods (How do instructors assess students’ work or performance?)

Demonstration of public speaking

Speech in writing

Grading Rubric

Grading Rubric

Grade Book Audit

Survey on student time management skills

Grade Book Record

Survey Instrument

Creating a Rubric

A rubric is a systematic scoring guide to evaluate student performance.

Three Essential Parts of a Rubric:1) A Set of Indicators of Learning2) A Scale3) A Set of Descriptors

Creating a Rubric

Indicators for a rubric-based assignment (essay, project, speech, performance) should be aligned with the instructional objectives. If an instructor’s objectives of teaching an essay are related to helping students compose an essay that is focused, coherent, organized, etc., then these features should serve as indictors of learning.

Example of Indicators for an Essay for an English Class: Focus Coherence Organization Sentence Structure Word Choice

Creating a Rubric

A scale indicates points to be assigned in scoring a piece of work on a continuum of quality. High numbers are typically assigned to the best work.

Scale examples: Needs Improvement (1)... Satisfactory (2)... Good (3)... Exemplary (4)

Beginning (1)... Developing (2)... Accomplished (3)... Exemplary (4)

Needs work (1)... Good (2)... Excellent (3)

Novice (1)... Apprentice (2)... Proficient (3)... Distinguished (4)

Creating a Rubric

Descriptors are used to describe signs of performance at each level; the description needs to include both strengths and weaknesses (Weaknesses should be described particularly in lower levels of performance).

Examples of Descriptors:Word Choice4-- Vocabulary reflects a thorough grasp of the language appropriate to the audience. Word choice is precise, creating a vivid image. Metaphors and other such devices may be used to create nuanced meaning.

3-- Vocabulary reflects a strong grasp of the language appropriate to the audience. Word choice is accurate, but may be inappropriate in a couple of places.

2-- Vocabulary reflects an inconsistent grasp of the language and may be inaccurate or inappropriate to the audience.

1-- Vocabulary is typically inaccurate and inappropriate to the audience. Word choice may include vague, non-descriptive, and/or trite expressions.

Example of a Rubric (A Snapshot)

Critical Thinking VALUE Rubric (AACC)

IndicatorsCapstone

4Milestones

3 2Benchmark

1Explanation of Issues Issue/problem to be

considered critically isstated clearly and describedcomprehensively, delivering all relevantinformation necessary for full understanding.

Issue/problem to be considered critically isstated, described, and clarified so thatunderstanding is not seriously impeded byomissions.

Issue/problem to be considered critically isstated but description leaves some termsundefined, ambiguities unexplored,boundaries undetermined, and/or backgrounds unknown.

Issue/problem to be considered critically is stated without clarification or description.

Evidence Information is taken from source(s) withenoughInterpretation/evaluation to developa comprehensive analysis or synthesis.

Information is taken from source(s) withenough interpretation/evaluation to developa coherent analysis or synthesis.

Information is taken from source(s) withsome interpretation/evaluation, but notenough to develop a coherent analysis orsynthesis.

Information is taken from source(s) without any interpreta-tion/evaluation.

Setting Criteria for Meeting SLO

To determine whether students have successfully met the learning outcomes, we need to specify what score or rating an individual student needs to get in order to be considered as meeting the learning outcome.

Examples:Measure: Exam on the history and purpose of Physical TherapyEvaluation Method: Score Weighting Guide with scores assigned to each exam questionCriterion for Meeting the SLO: A score of 70 or above on the exam

Measure: Essay describing the scope of physical therapyEvaluation Method: Grading Rubric specifying levels of quality in identified areas Criterion for Meeting the SLO: A score of 3 or above on a scale of 1-4

Specifying Target Outcomes

To determine whether students as a group have successfully met the learning outcomes, we need to specify what % from the group should meet the learning outcome.

Example I:Measure: Exam on the history and purpose of Physical TherapyEvaluation Method: Score Weighting Guide with scores assigned to each exam questionCriterion for Meeting the SLO: A score of 70 or above on the exam

Target Outcome: 85% of the students will score 70 or above on the exam.

Specifying Target Outcomes

To determine whether students as a group have successfully met the learning outcomes, we need to specify what % from the group should meet the learning outcome.

Example II:Measure: Essay describing the scope of physical therapyEvaluation Method: Grading Rubric specifying levels of quality in identified areas Criterion for Meeting the SLO: A score of 3 or above on a scale of 1-4

Target Outcome: 70% of the students will score a 3 or above on their essays

Recap: SLO Assessment Plan

Student Learning Outcome Assessment PlanMission:

PLO or CLO Statement

Measure (Student Work/Performance)

Teacher's Evaluation Method

Criterion(What cut-off score is considered as meeting PLO or CLO for each student?)

Target Outcome (Group Success: % of the group meeting the PLO or CLO)

PLO Assessment Plan: (Example)

Associate Degree in Political Science Program Learning OutcomesMission: The Political Science Department is committed to producing students who are knowledgeable about the operation of Texas and national government. Students should also have the ability to think critically about important political topics that will prepare them for transfer to higher institutions of learning and provide a framework for lifetime learning skills.

PLO Statement Measure (Student Work/Performance)

Teacher's Evaluation Method

Criterion(What cut-off score is considered as meeting PLO for each student?)

Target Outcome (Group Success: % of the group meeting the PLO)

Students will understand the American politics subfield of political science.

GOVT2304 (Intro to Political Science) Essay Question on American Politics

Essay Assessment Rubric A score of 3 or above on essay (on a scale of 1-5)

80% of the students answering the essay question will get a score 3 or above.

Students will understand the political theory subfield of political science.

GOVT2304 Essay Question on Political Theory

Essay Assessment Rubric A score of 3 or above on essay (on a scale of 1-5)

80% of the students answering the essay question will get a score 3 or above.

Students will demonstrate a level of civic engagement

Internship Project Comments on strengths and weaknesses of the students’ internship performance in an evaluation letter

70% of the comments in the evaluation letter are positive.

95% of the students will get a positive evaluation letter from their internship supervisor.

CLO Assessment Plan: (Example)

Engl 1302 Rhetoric and Composition Course Learning Outcome Assessment PlanCLO Statement Measure (Student

Work/Performance) Teacher's Evaluation Method

Criterion(What cut-off score is considered as meeting CLO for each student?)

Target Outcome (Group Success: % of the group meeting the CLO)

Students will be able to present a valid argument using convincing evidence.

Persuasive Essay Essay Grading Rubric A score of 3 or above (1-5 scale)

80% of the students taking the course will receive a score of 3 or above.

Students will be able to demonstrate appropriate use of standard English.

Grammar QuizWriting Portfolio

a. Quiz grading standardb. Portfolio Grading Rubric

a. A score of 70 on the quizb. A score of 3 or above (1-5 scale)

75% of the students taking the Engl1302 course will receive a score of 70 for the quiz and a score of 3 or above for the portfolios.

Student will apply researched information in writing a research paper.

Summary of ReadingsResearch Paper

a. Summary Grading Rubricb. Research Paper Grading Rubric

a. A score of 3 or above (1-5 scale)b. A score of 3 or above (1-5 scale)

75% of the students taking the Engl1302 course will receive a score of 3 or above for the summary assignment and for the research papers.

Do’s and Don’ts When Creating an

SLO Assessment PlanDo’s1. When creating PLO or CLO statements, DO include both

lower level and higher level of cognitive skills: not only knowledge, comprehension, but also application, analysis, evaluation, and synthesis.

2. When selecting assessment measures for PLOs, DO include both course-embedded assignments and program level measures (thesis, exit exams, or national exams, or graduate portfolios, or internship project, or capstone course project).

3. Encourage faculty participation in the design of the Assessment Plan.

4. Try to use common assessment assignments or tools, but allow for flexibility.

Do’s and Don’ts When Creating an

SLO Assessment PlanDon’ts

Don’t be overly ambitious—Create only 5 to 7 PLOs or CLOs, and use only 7-10 pieces of data!

Academy for Planning, Assessment, and Research