TerminologyThe Black BoxTimetablesData/MeasurementData/Measurement CautionsProgram Outcomes & ExamplesStudent Outcomes & ExamplesClosing & Questions
Actions leading to specific, observable results
What students can expect to learn How students or programs will be
different as a result of what happens in the box
What knowledge, skills, and abilities will be improved
What measurable changes occur in tutoring programs
Planning for Assessment Continual processes Multiple times per semester Once or twice per semester Two or three times per year Once per year Every two years Every 4 years
How to measure Program-generated data Course-specific data Campus data Campus surveys Standardized tools Locally developed tools
More about locally developed tools Surveys Likert-type scales Explicated scales Behavioral rubrics Self assessment by student Tutor assessment Observer assessment
Direct vs. Indirect MeasuresData access and consistencyRealistic limitationsCooperationThe Baseline ChallengeTime & money
(and staff and comfort and ……)
Possible Areas of Assessment Focus Establishing baselines Usage Satisfaction or quality Level/variety of services Staff development Reaching certification or meeting
standards
Outcome Statement As a result of the new marketing strategy,
the number of students attending group tutoring sessions will increase by 10% in Spring 2011.
Timetable One semester
Starting Measure (Benchmark) Average of past attendance
Ending Measure Workshop attendance records
Outcome Statement During 2010-11, the number of tutors who
earn CRLA Level I certification will increase by 5%.
Timetable One academic year
Starting Measure (Benchmark) Number of tutors achieving Level I in previous
year(s) Ending Measure
Number of tutors achieving Level I in 2010-11
Outcome Statement Outside reviewers using the NADE self-
study process will score the tutoring program at 85% or above on all sections.
Timetable One year
Starting Measure (Benchmark) Internal review & self-study
Ending Measure External review
Outcome Statement As a result of tutor training sessions, tutors
will demonstrate an improved understanding of Bloom’s Taxonomy.
Timetable One semester or year, depending on
training schedule Starting Measure (Benchmark)
Pre-test Ending Measure
Post-test
TYPES
Affective Build confidence Improve attitude Reduce anxiety
Cognitive Mastery of material Conscious strategies
use
MEASURES
Qualitative Interviews & focus groups Opinion surveys Observations or self
reports
Quantitative Assignment & course
grades Instrument scores Observation rubrics
Outcome Statement After being tutored for at least one hour,
clients will be more confident in their ability to accomplish the academic task they focused on in tutoring.
Timetable One tutoring session
Starting Measure (Benchmark) Client self-report
Ending Measure Client self report
Outcome Statement Students who work with the study skills
tutors will demonstrate an increased ability to use memorization strategies.
Timetable One tutoring session
Starting Measure (Benchmark) Tutor or observer checklist
Ending Measure Tutor or observer checklist
Outcome Statement 75% of the students who work with a
reading tutor for at least two sessions will score at least 75% on the main ideas reading test.
Timetable Within three weeks
Starting Measure (Benchmark) Reading quiz at start of tutoring
Ending Measure Chapter reading test in class
Outcome Statement Students who attend at least 10 group
tutoring sessions for Algebra Review 090 will earn an average grade of B or above in the course.
Timetable One semester, length of course
Starting Measure (Benchmark) Placement score into course
Ending Measure Course grade
Outcome Statement After attending mandatory tutoring for all 5
of their developmental education courses, students will be retained until the start of their junior year at a rate at least 25% higher than developmental students for whom tutoring was not required.
Timetable Two years
Starting Measure (Benchmark) Developmental course placement
Ending Measure Enrollment level
Outcome Statement Students attending SI for World Religions
will “[apply] previously understood information, concepts, and experiences to a new situation or setting.” (CAS)
Timetable Within one semester
Starting Measure (Benchmark) Pre-survey
Ending Measure Post-survey
Check with your institutional research or assessment office
Review early drafts of outcomes statements with your supervisor
Pre-test any locally created measurement tool
Be prepared for outcomes that are not met or wildly exceeded
Remember the black box: we do not own or control the box, and our ability to see into it is very limited