student learning and program outcomes applicable to disability offices? 2015 ahead conference july...
TRANSCRIPT
Student Learning andProgram Outcomes
Applicable to Disability Offices?2015 AHEAD Conference
July 15, St. Paul, Minnesota
Presenters• Jean Ashmore
• Past President AHEAD• current AHEAD representative to CAS• Rice University Disability Director Emerita
• Karen Agee• Past President College Reading & Learning Association• Univ. of Northern Iowa, Reading & Learning Coord., Emerita
• Paul Harwell• Texas A&M University
• Tom Thompson• Past AHEAD Board Member• Harper College Disability Director Emeritus
Lunch & Learn Session Topics
• Introducing CAS student learning and development outcomes
• Linking student outcomes and program standards
• Developing systematic practices for assessing program outcomes
• Managing outcome data
• Demonstrating continuous improvement and accreditation requirements
Why is DS self-assessment critically important?
• Gathering formative and summative data (metrics) is a primary means of:- Understanding what is working well
- Zeroing in on how to improve services- Justifying additional resources
• Demonstrating the impact of DS on student life, growth and success- Answering how do these supports impact engagement, persistence and completion
CAS - Counsel for Advancement of Standards in Higher Education
• www.cas.edu
• 47 sets of Standards & Guidelines for DRS and other functions usually within Student Affairs
• Disability Resources & Services standards revised in 2013 (Karen Agee, chair; Jean Ashmore, expert)
• Mission and Program the first of 12 areas of self-assessment
• CAS member associations include AHEAD, CRLA, NACADA, and 38 other professional organizations
Program Outcomes
• Defined strategically to be measurable
• Linked to DS mission, program of services, and resources – all locally developed
• Applicable– Within the DS office, e.g., tech training, testing,
alternative format production– Across campus, e.g., trainings, outreach, lectures
CAS Student Learning & Development Outcomes
• Adopted by CAS in 2008 (in current form):http://www.cas.edu/learningoutcomes• Based on Learning Reconsidered 2: Implementing
a Campus-Wide Focus on the Student Experience (Keeling, 2006)
• Applicable, in part or whole, to all 47 areas of CAS standards
• Expected to be defined locally and to be measured directly and indirectly
SL&D Outcomes
• Student Outcome Domains1. Knowledge acquisition, construction, integration
and application
2. Cognitive complexity
3. Intrapersonal development
4. Interpersonal competence
5. Humanitarianism and civic engagement
6. Practical competence
Dimensions
• Each of the 6 Outcome Domains has Dimensions:– Example – SLO domain of Interpersonal Competence has
dimensions Meaningful Relationships and Interdependence
– Example – SLO domain of Practical Competence has dimensions Pursuing goals, Communicating effectively, Technological competence, Managing personal affairs,
Managing career development, Demonstrating professionalism, Maintaining health & wellness, Living a purposeful & satisfying life
• Not all domains will apply to each campus entity
Domains and Dimensionshttp://www.cas.edu/learningoutcomes
• Domain - Practical CompetenceDimensions
Communicating effectively -Disclosing & articulating about one’s use of
accommodationsTechnical competence - Using technology
effectivelyManaging personal affairs - Balancing personal
life and college demandsManaging career development - Preparing for
transition to employment or grad school – issues of disclosure & using accommodations
How can SLOs be used by DS?
• Step 1: Define one’s DS mission and program of activities
• Step 2: Review SLOs critically, looking for links to program services & activities
• Step 3: Write specific student outcomes that are measurable, directly & indirectly
• Step 4: Collect outcome data regularly, and analyze, report, and adjust program services accordingly
How can SLOs be used by DS?
• Assessing SLOs formalizes what every DS professional does all the time anyway, to answer questions like these:– How well did that session go?– What did those students think our session accomplished?– Are we providing good services?– Do student employees know appropriate strategies?
• Assessing program outcomes formalizes the bigger questions we hope to affect:o Are we retaining our clients at this institution?o Are our clients academically successful here?
Methods of Assessing SLOs?Maki, P. L. (2004). Assessing for learning: Building a sustainable commitment across the institution. Sterling, VA: Stylus.
Direct methods• Students demonstrate learning via some form of standardized test focusing on
aspects of student learningExamples: CAAP, CLA, ETS Proficiency Profile (formerly MAPP), GRE subject tests, PRAXIS exams
Indirect methods• Students report perceptions of their learning and the educational environment
that supports that learningExamples: CSEQ, NSSE, NSLLP
Performance-based methods• Students represent learning in response to assignments or projects that are
embedded in their educational experiences• Examples: Successful solution of a math problem not previously understood,
demonstrated writing skill, panic replaced by confidenceBut we DS professionals can create more direct methods for assessing learning than
standardized tests!
Examples of SLOs within DS
Domain 1: Knowledge Dimension: Relating knowledge to daily life
• Students who complete a DS intake workshop will
demonstrate an increase of at least 50% in knowledge
of the responsibilities of students and DS as
demonstrated by scores on pre- and post-workshop
quiz
Examples of SLOs within DS
Domain 1: KnowledgeDimension: Relating knowledge to daily life
• Students participating in at least three individual DS
sessions in the semester will demonstrate 50% improved
knowledge of accommodations appropriate for their
disability or disabilities as measured by their score on the
assessment checklist administered by DS staff
Examples of SLOs within DS
Domain 2: Cognitive complexityDimension: Critical thinking
• Students who work with a DS staff member to identify and acquire accommodations for a disability will, at the end of the session, be able to articulate understanding of the nature of his disability barriers, appropriate accommodations agreed upon, and steps the student has decided to take to acquire those accommodations
Examples of SLOs within DS
Domain 6: Practical Competence Dimension: Demonstrating professionalism
• By the 10th hour of training, student staff will
demonstrate development of professionalism as
measured by the program’s professionalism rubric
created during the 2nd hour of training
Examples of SLOs within DS
Domain 6: Practical Competence Dimensions: Communication Competence Pursuing Goals
• Students in group DS coaching sessions will demonstrate communication and goal-setting skills learned in sessions #2-3 by writing and emailing to the DS session leader and student members of the group a statement of academic goals with no more than 3 errors of spelling, grammar, and punctuation
Examples of SLOs within DS
Domain 6: Practical Competence Dimension: Managing personal affairs
• Students will demonstrate use of a time management system on Day 8 of the 12-session Effective Study Strategies course as measured by instructor analysis of in-class presentations, documented by artifacts [such as calendar, electronic schedule, and completed assignments]
Examples of SLOs within DS
Domain 6: Practical CompetenceDimensions: Pursuing goals
Managing personal affairs
Students who work with a DS staff member on
strategies for self-advocacy will document successful
attempts to achieve agreed-upon goals at the end-
of-semester session.
Why SLOs in DS?
• The purpose of setting and measuring student
learning & development outcomes in DS is
to build excellent programs
• Growth in DS #s = DS expansion = new
resources
• All driven by outcome data
DS Program Outcomes• Concept: Assess and evaluate overall program
of disability resources & services• Foundation: Have a strategic plan for DS with
identified goals and an operational plan with specific short range goals
• Regularly collect input from constituencies:– Students– Staff– Faculty– Institutional partners
Examples of Program Outcomes
• DS example Strategic Goal: “Promote systemic improvement in campus accessibility for facilities, digital systems and campus procedures”– FY Outcome – Build alt format production into the
course development process and train 12 faculty• DS example Strategic Goal: “Provide consultation
and training to faculty, staff and students that results in greater accessibility”– FY Outcome – Create a course substitution process
with support of the faculty, administration and DS.
Examples of Program Outcomes merged with SLOs
• Program Outcome – Interpreter and transcription services will be used appropriately and be cost effective
• SLOs – Practical Competence & Interpersonal Competence -students using interpreters or transcriptionists will articulate the guidelines for these services and follow required notification timelines with 95% compliance
Sample Assessment Schedule for a DS Program
• See Appendix A
Managing Outcome Data
• Develop comprehensive data collection practices
• Develop routine reporting cycles and distribution
• Use data as evidence of program’s budget needs– Key point here – doing assessment and evaluation
takes time and skill. In smaller offices, if a Director is to do this work, other staff are needed to see students
Continuous Improvement & Accreditation
• 360 degree thinking – data leads to action plan = continuous assessment = continuous improvement
• Accreditation agencies expect
– Evidence of effectiveness from both academic and co-curricular programs
– Strategic plans with goals and objectives and program assessment for continuous improvement
– Demonstration by disability offices of value to students
Conclusion: Disability programs
• must think strategically!
• need to develop measurable student learning outcomes!
• must incorporate program assessment into all elements of DS activity on campus as good business practice!
• need to go in this direction before being forced to by those who do not understand our work!
Contact Information
• Karen Agee– [email protected]
• Jean Ashmore– [email protected]
• Paul Harwell– [email protected]
• Tom Thompson– [email protected]