an integrated model for assessing course and program slos david a. wood, jr., ph.d. associate...

42510011 0010 1010 1101 0001 0100 1011

An Integrated Model for Assessing Course and

Program SLOs

David A. Wood, Jr., Ph.D.Associate Professor of Astronomy

San Antonio College

4251

0011 0010 1010 1101 0001 0100 1011

Mapping Student Learning Outcomes to Competencies

• SACS & THECB Mandates

• Developing a Single Assessment Model

• Integrating Assessment into the Core Curriculum

• What About Those Pesky ACGM Outcomes?

• eLumen

• The Future of Assessment

• Key Terms

42510011 0010 1010 1101 0001 0100 1011

SACS & THECB Mandates

4251

0011 0010 1010 1101 0001 0100 1011

Era of Accountability

• SACS Requirement 3.3.1.1:

The institution identifies expected outcomes, assesses the extent to which it achieves these outcomes, and provides evidence of improvement based on analysis of the results in educational programs, to include student learning outcomes.

(Institutional Effectiveness)

4251

0011 0010 1010 1101 0001 0100 1011

Era of Accountability

• SACS Requirement 3.5.1:

The institution identifies college-level general education competencies and the extent to which students have attained them.

(General education competencies)

4251

0011 0010 1010 1101 0001 0100 1011

Era of Accountability

• THECB – New Core Curriculum:

“Institutions will continue the assessment practices required by SACSCOC. Institutions will assess the six core objectives using these practices and submit the report to the Coordinating Board every ten years on a schedule in accord with their accreditation reaffirmation schedule. Accountability for assessment of the core objectives is at the institutional level.”

http://www.thecb.state.tx.us/index.cfm?objectid=42E67B6B-002A-90EE-C17A4779C473E964

4251

0011 0010 1010 1101 0001 0100 1011

Era of Accountability

• THECB – Spring 2014 ACGM (p. 4):

When offering the courses, institutions must include all topics in the ACGM description and provide instruction to cover and assess all of the learning outcomes. Institutions may not delete any topics in the course descriptions or any of the student learning outcomes as provided in the ACGM. Based on local needs, an institution may include additional topics and learning outcomes.

4251

0011 0010 1010 1101 0001 0100 1011

Era of Accountability

• Alamo Colleges Board Policy A.1.3:

Strategic Objective II: LeadershipProvide opportunities for Alamo Colleges students and employees to develop as principle-centered leaders.

–Goal A. Incorporate personal and social responsibility, global citizenship, critical thinking, and life-long learning as the framework of principle-centered leadership into the culture and curriculum of the Alamo Colleges.

4251

0011 0010 1010 1101 0001 0100 1011

Assessment Whirlwind!• SACS 3.5.1 requires us to measure

program SLOs;

SACS 3.5.1 requires us to measure general education competencies;

THECB requires us to measure the state core curriculum core objectives;

ACGM requires us to measure state-mandated course SLOs.

Alamo Colleges Board Policy A.1.3 requires us to measure Leadership skills.

• Five different assessment requirements!

42510011 0010 1010 1101 0001 0100 1011

Developing A Single Assessment Model

4251

0011 0010 1010 1101 0001 0100 1011

Building an Assessment Model

• In 2009, SAC began requiring every instructor to submit course SLO attainment reports for every student.

• Course SLOs were (haphazardly) mapped to college competencies.

• College aggregated results from all courses to get coarse measures of program SLO and general education competency attainment.

4251

0011 0010 1010 1101 0001 0100 1011

The Measure of a Course

4251

0011 0010 1010 1101 0001 0100 1011

Did Students REALLY Learn?• Simple outcome achievement record

4251

0011 0010 1010 1101 0001 0100 1011

Taming the Whirlwind

4251

0011 0010 1010 1101 0001 0100 1011

Building an Assessment Model

• Advent of new state core curriculum caused us to revise the college’s general education competencies

• In Fall 2013, SAC adopted the state Core Objectives as a basis for new competencies.

• ~70% of credit hours applied toward an Associate degree is core curriculum, so general education competencies can be effectively written as program SLOs.

4251

0011 0010 1010 1101 0001 0100 1011

THECB Core Objectives

4251

0011 0010 1010 1101 0001 0100 1011

New College Competencies

(1) Critical Thinking

(2) Communication Skills

(3) Quantitative and Scientific Literacy

(4) Teamwork

(5) Social Responsibility

(6) Personal Responsibility

(7) Leadership

(8) Performance

4251

0011 0010 1010 1101 0001 0100 1011

4251

0011 0010 1010 1101 0001 0100 1011

Taming the Whirlwind

42510011 0010 1010 1101 0001 0100 1011

Integrating Assessment into the Core Curriculum

4251

0011 0010 1010 1101 0001 0100 1011

SLO → Competency Mapping

4251

0011 0010 1010 1101 0001 0100 1011

4251

0011 0010 1010 1101 0001 0100 1011

4251

0011 0010 1010 1101 0001 0100 1011

4251

0011 0010 1010 1101 0001 0100 1011

4251

0011 0010 1010 1101 0001 0100 1011

4251

0011 0010 1010 1101 0001 0100 1011

4251

0011 0010 1010 1101 0001 0100 1011

4251

0011 0010 1010 1101 0001 0100 1011

4251

0011 0010 1010 1101 0001 0100 1011

Competency Tab

4251

0011 0010 1010 1101 0001 0100 1011

Competency Tab

4251

0011 0010 1010 1101 0001 0100 1011

Summary Tab

42510011 0010 1010 1101 0001 0100 1011

What about those Pesky ACGM Outcomes?

4251

0011 0010 1010 1101 0001 0100 1011

Handling ACGM Outcomes

• The ACGM now contains generic course SLOs for many discipline rubrics, and many of these outcomes are difficult to measure.

• One alternative is to adopt the ACGM SLOs as the course SLOs and then add local SLOs to the list.– Problem: Puts burden on faculty member to force fit assessments or

revamp a course.

• A better alternative is to treat ACGM course SLOs as de facto “Focus Areas”; then map to them using local course SLOs.

4251

0011 0010 1010 1101 0001 0100 1011

Taming the Whirlwind

42510011 0010 1010 1101 0001 0100 1011

eLumen

4251

0011 0010 1010 1101 0001 0100 1011

eLumen Solution• Excel matrices were cumbersome for faculty

and challenging for IR staff.

• eLumen Collaborative provides an effective solution handling back end work of mapping course SLOs to program SLOs and competencies.

• Tools in eLumen track student learning and disaggregate results in many different ways.

• Closes the loop by linking SLO results back to strategic plan.

• Handles all types of SLO assessment, not just per-student assessment.

http://elumen.info

4251

0011 0010 1010 1101 0001 0100 1011

4251

0011 0010 1010 1101 0001 0100 1011

4251

0011 0010 1010 1101 0001 0100 1011

4251

0011 0010 1010 1101 0001 0100 1011

42510011 0010 1010 1101 0001 0100 1011

The Future of Assessment

4251

0011 0010 1010 1101 0001 0100 1011

Future of Assessment

• Competency-based learning We can know much more about a student’s skills than just her average

performance in a course. Course-embedded assessment (e.g., Khan Academy approach) instead of

assessment day “fire drills.” Per-student assessment will replace random sampling or aggregated

reporting as technology is more capable of supporting granular reporting.

• Longitudinal tracking of students Track student performance throughout their educational career. Improve retention, productive grades, and other momentum point measures. Able to identify and address leaks in the educational pipeline based upon

demographic factors.

4251

0011 0010 1010 1101 0001 0100 1011

Final Thoughts

• Assessment should be a fundamentally human endeavor. People are not one-dimensional and cannot be boiled down to a single number.

• Assessment is not about fulfilling an accreditation requirement; it is about improving student learning.

• Competency-based education and longitudinal tracking are coming. If faculty do not embrace and control assessment, then computers and bubble forms will.

4251

0011 0010 1010 1101 0001 0100 1011

Resources

(1) American Association of Colleges & Universities VALUE Project Rubrics (http://www.aacu.org/value/index.cfm)

(2) Amarillo College Competency Information & Rubrics (http://www.actx.edu/iea/index.php?module=article&id=67)

(3) Secretary’s Commission on Achieving Necessary Skills (http://wdr.doleta.gov/SCANS/)

(4) National Institute for Learning Outcomes Assessment (http://www.learningoutcomesassessment.org/)

42510011 0010 1010 1101 0001 0100 1011

Thank You!

Questions?

David A. Wood, Jr., Ph.D.

dwood30@alamo.edu

(210) 486-0063

42510011 0010 1010 1101 0001 0100 1011

Key Terms

4251

0011 0010 1010 1101 0001 0100 1011

Key Terms THECB Core Objectives

Six broad skills - Critical Thinking, Communication Skills, Empirical and Quantitative Skills, Teamwork, Social Responsibility, and Personal Responsibility – that must be integrated into all undergraduate core curricula at Texas colleges and universities.

THECB Curriculum Areas

Major divisions of general education fields of study – Communication; Mathematics; Life & Physical Sciences; Language, Philosophy, and Culture; Creative Arts; American History; Government/Political Science; Social/Behavioral Science – as defined by the THECB.

4251

0011 0010 1010 1101 0001 0100 1011

Key Terms General Education Competencies

A set of broad skills, - to include Critical Thinking, Communication Skills, Empirical and Quantitative Skills, Teamwork, Social Responsibility, Personal Responsibility, Leadership, and Performance - that graduates from SAC are expected to attain.

Competency Focus Areas

Components of general education competencies that divide the scope of a specific competency into several more narrowly-defined concepts. They are a bridge for mapping specific course and program learning outcomes to general education competencies.

4251

0011 0010 1010 1101 0001 0100 1011

Key Terms

SLO Mapping Categories

Three categories that determine how strongly individual, course, and program learning outcomes engage a competency.

Introductory – A learning outcome, course, or program minimally touches a general education competency, usually in just a single Focus Area.

Reinforcement – A learning outcome, course, or program engages multiple Focus Areas of a general education competency in some breadth and/or depth.

Comprehensive - A learning outcome, course, or program engages all Focus Areas of a general education competency in significant breadth and depth.