an integrated model for assessing course and program slos david a. wood, jr., ph.d. associate...
TRANSCRIPT
42510011 0010 1010 1101 0001 0100 1011
An Integrated Model for Assessing Course and
Program SLOs
David A. Wood, Jr., Ph.D.Associate Professor of Astronomy
San Antonio College
4251
0011 0010 1010 1101 0001 0100 1011
Mapping Student Learning Outcomes to Competencies
• SACS & THECB Mandates
• Developing a Single Assessment Model
• Integrating Assessment into the Core Curriculum
• What About Those Pesky ACGM Outcomes?
• eLumen
• The Future of Assessment
• Key Terms
42510011 0010 1010 1101 0001 0100 1011
SACS & THECB Mandates
4251
0011 0010 1010 1101 0001 0100 1011
Era of Accountability
• SACS Requirement 3.3.1.1:
The institution identifies expected outcomes, assesses the extent to which it achieves these outcomes, and provides evidence of improvement based on analysis of the results in educational programs, to include student learning outcomes.
(Institutional Effectiveness)
4251
0011 0010 1010 1101 0001 0100 1011
Era of Accountability
• SACS Requirement 3.5.1:
The institution identifies college-level general education competencies and the extent to which students have attained them.
(General education competencies)
4251
0011 0010 1010 1101 0001 0100 1011
Era of Accountability
• THECB – New Core Curriculum:
“Institutions will continue the assessment practices required by SACSCOC. Institutions will assess the six core objectives using these practices and submit the report to the Coordinating Board every ten years on a schedule in accord with their accreditation reaffirmation schedule. Accountability for assessment of the core objectives is at the institutional level.”
http://www.thecb.state.tx.us/index.cfm?objectid=42E67B6B-002A-90EE-C17A4779C473E964
4251
0011 0010 1010 1101 0001 0100 1011
Era of Accountability
• THECB – Spring 2014 ACGM (p. 4):
When offering the courses, institutions must include all topics in the ACGM description and provide instruction to cover and assess all of the learning outcomes. Institutions may not delete any topics in the course descriptions or any of the student learning outcomes as provided in the ACGM. Based on local needs, an institution may include additional topics and learning outcomes.
4251
0011 0010 1010 1101 0001 0100 1011
Era of Accountability
• Alamo Colleges Board Policy A.1.3:
Strategic Objective II: LeadershipProvide opportunities for Alamo Colleges students and employees to develop as principle-centered leaders.
–Goal A. Incorporate personal and social responsibility, global citizenship, critical thinking, and life-long learning as the framework of principle-centered leadership into the culture and curriculum of the Alamo Colleges.
4251
0011 0010 1010 1101 0001 0100 1011
Assessment Whirlwind!• SACS 3.5.1 requires us to measure
program SLOs;
SACS 3.5.1 requires us to measure general education competencies;
THECB requires us to measure the state core curriculum core objectives;
ACGM requires us to measure state-mandated course SLOs.
Alamo Colleges Board Policy A.1.3 requires us to measure Leadership skills.
• Five different assessment requirements!
42510011 0010 1010 1101 0001 0100 1011
Developing A Single Assessment Model
4251
0011 0010 1010 1101 0001 0100 1011
Building an Assessment Model
• In 2009, SAC began requiring every instructor to submit course SLO attainment reports for every student.
• Course SLOs were (haphazardly) mapped to college competencies.
• College aggregated results from all courses to get coarse measures of program SLO and general education competency attainment.
4251
0011 0010 1010 1101 0001 0100 1011
The Measure of a Course
4251
0011 0010 1010 1101 0001 0100 1011
Did Students REALLY Learn?• Simple outcome achievement record
4251
0011 0010 1010 1101 0001 0100 1011
Taming the Whirlwind
4251
0011 0010 1010 1101 0001 0100 1011
Building an Assessment Model
• Advent of new state core curriculum caused us to revise the college’s general education competencies
• In Fall 2013, SAC adopted the state Core Objectives as a basis for new competencies.
• ~70% of credit hours applied toward an Associate degree is core curriculum, so general education competencies can be effectively written as program SLOs.
4251
0011 0010 1010 1101 0001 0100 1011
THECB Core Objectives
4251
0011 0010 1010 1101 0001 0100 1011
New College Competencies
(1) Critical Thinking
(2) Communication Skills
(3) Quantitative and Scientific Literacy
(4) Teamwork
(5) Social Responsibility
(6) Personal Responsibility
(7) Leadership
(8) Performance
4251
0011 0010 1010 1101 0001 0100 1011
4251
0011 0010 1010 1101 0001 0100 1011
Taming the Whirlwind
42510011 0010 1010 1101 0001 0100 1011
Integrating Assessment into the Core Curriculum
4251
0011 0010 1010 1101 0001 0100 1011
SLO → Competency Mapping
4251
0011 0010 1010 1101 0001 0100 1011
4251
0011 0010 1010 1101 0001 0100 1011
4251
0011 0010 1010 1101 0001 0100 1011
4251
0011 0010 1010 1101 0001 0100 1011
4251
0011 0010 1010 1101 0001 0100 1011
4251
0011 0010 1010 1101 0001 0100 1011
4251
0011 0010 1010 1101 0001 0100 1011
4251
0011 0010 1010 1101 0001 0100 1011
4251
0011 0010 1010 1101 0001 0100 1011
Competency Tab
4251
0011 0010 1010 1101 0001 0100 1011
Competency Tab
4251
0011 0010 1010 1101 0001 0100 1011
Summary Tab
42510011 0010 1010 1101 0001 0100 1011
What about those Pesky ACGM Outcomes?
4251
0011 0010 1010 1101 0001 0100 1011
Handling ACGM Outcomes
• The ACGM now contains generic course SLOs for many discipline rubrics, and many of these outcomes are difficult to measure.
• One alternative is to adopt the ACGM SLOs as the course SLOs and then add local SLOs to the list.– Problem: Puts burden on faculty member to force fit assessments or
revamp a course.
• A better alternative is to treat ACGM course SLOs as de facto “Focus Areas”; then map to them using local course SLOs.
4251
0011 0010 1010 1101 0001 0100 1011
Taming the Whirlwind
42510011 0010 1010 1101 0001 0100 1011
eLumen
4251
0011 0010 1010 1101 0001 0100 1011
eLumen Solution• Excel matrices were cumbersome for faculty
and challenging for IR staff.
• eLumen Collaborative provides an effective solution handling back end work of mapping course SLOs to program SLOs and competencies.
• Tools in eLumen track student learning and disaggregate results in many different ways.
• Closes the loop by linking SLO results back to strategic plan.
• Handles all types of SLO assessment, not just per-student assessment.
http://elumen.info
4251
0011 0010 1010 1101 0001 0100 1011
4251
0011 0010 1010 1101 0001 0100 1011
4251
0011 0010 1010 1101 0001 0100 1011
4251
0011 0010 1010 1101 0001 0100 1011
42510011 0010 1010 1101 0001 0100 1011
The Future of Assessment
4251
0011 0010 1010 1101 0001 0100 1011
Future of Assessment
• Competency-based learning We can know much more about a student’s skills than just her average
performance in a course. Course-embedded assessment (e.g., Khan Academy approach) instead of
assessment day “fire drills.” Per-student assessment will replace random sampling or aggregated
reporting as technology is more capable of supporting granular reporting.
• Longitudinal tracking of students Track student performance throughout their educational career. Improve retention, productive grades, and other momentum point measures. Able to identify and address leaks in the educational pipeline based upon
demographic factors.
4251
0011 0010 1010 1101 0001 0100 1011
Final Thoughts
• Assessment should be a fundamentally human endeavor. People are not one-dimensional and cannot be boiled down to a single number.
• Assessment is not about fulfilling an accreditation requirement; it is about improving student learning.
• Competency-based education and longitudinal tracking are coming. If faculty do not embrace and control assessment, then computers and bubble forms will.
4251
0011 0010 1010 1101 0001 0100 1011
Resources
(1) American Association of Colleges & Universities VALUE Project Rubrics (http://www.aacu.org/value/index.cfm)
(2) Amarillo College Competency Information & Rubrics (http://www.actx.edu/iea/index.php?module=article&id=67)
(3) Secretary’s Commission on Achieving Necessary Skills (http://wdr.doleta.gov/SCANS/)
(4) National Institute for Learning Outcomes Assessment (http://www.learningoutcomesassessment.org/)
42510011 0010 1010 1101 0001 0100 1011
Thank You!
Questions?
David A. Wood, Jr., Ph.D.
(210) 486-0063
42510011 0010 1010 1101 0001 0100 1011
Key Terms
4251
0011 0010 1010 1101 0001 0100 1011
Key Terms THECB Core Objectives
Six broad skills - Critical Thinking, Communication Skills, Empirical and Quantitative Skills, Teamwork, Social Responsibility, and Personal Responsibility – that must be integrated into all undergraduate core curricula at Texas colleges and universities.
THECB Curriculum Areas
Major divisions of general education fields of study – Communication; Mathematics; Life & Physical Sciences; Language, Philosophy, and Culture; Creative Arts; American History; Government/Political Science; Social/Behavioral Science – as defined by the THECB.
4251
0011 0010 1010 1101 0001 0100 1011
Key Terms General Education Competencies
A set of broad skills, - to include Critical Thinking, Communication Skills, Empirical and Quantitative Skills, Teamwork, Social Responsibility, Personal Responsibility, Leadership, and Performance - that graduates from SAC are expected to attain.
Competency Focus Areas
Components of general education competencies that divide the scope of a specific competency into several more narrowly-defined concepts. They are a bridge for mapping specific course and program learning outcomes to general education competencies.
4251
0011 0010 1010 1101 0001 0100 1011
Key Terms
SLO Mapping Categories
Three categories that determine how strongly individual, course, and program learning outcomes engage a competency.
Introductory – A learning outcome, course, or program minimally touches a general education competency, usually in just a single Focus Area.
Reinforcement – A learning outcome, course, or program engages multiple Focus Areas of a general education competency in some breadth and/or depth.
Comprehensive - A learning outcome, course, or program engages all Focus Areas of a general education competency in significant breadth and depth.