assessing co-curricular learning jonathan keiser senior officer, oasis, the higher learning...
TRANSCRIPT
![Page 1: Assessing Co-curricular Learning Jonathan Keiser Senior Officer, OASIS, The Higher Learning Commission Robert Mundhenk Assessment Consultant](https://reader036.vdocument.in/reader036/viewer/2022062717/56649e305503460f94b206e9/html5/thumbnails/1.jpg)
Assessing Co-curricularAssessing Co-curricularLearningLearning
Jonathan KeiserSenior Officer, OASIS, The Higher Learning Commission
Robert MundhenkAssessment Consultant
![Page 2: Assessing Co-curricular Learning Jonathan Keiser Senior Officer, OASIS, The Higher Learning Commission Robert Mundhenk Assessment Consultant](https://reader036.vdocument.in/reader036/viewer/2022062717/56649e305503460f94b206e9/html5/thumbnails/2.jpg)
Traditional Approaches to Traditional Approaches to Assessment of LearningAssessment of Learning
Ask Institutional Research about graduation, retention, GPAs, and the like
Ask faculty about their teaching and the learning it produces—but not necessarily whether they know they’re producing learning
Ask faculty how they know they’re producing learning
![Page 3: Assessing Co-curricular Learning Jonathan Keiser Senior Officer, OASIS, The Higher Learning Commission Robert Mundhenk Assessment Consultant](https://reader036.vdocument.in/reader036/viewer/2022062717/56649e305503460f94b206e9/html5/thumbnails/3.jpg)
The Institutional MismatchThe Institutional Mismatch
Traditional wisdom: Learning outcomes need to be aligned at course, program, and institutional levels
But where are many general education goals, like “tolerance” and “teamwork” and “the ability to function in an increasingly diverse world” taught and assessed?
Or is “taught” the right word???
![Page 4: Assessing Co-curricular Learning Jonathan Keiser Senior Officer, OASIS, The Higher Learning Commission Robert Mundhenk Assessment Consultant](https://reader036.vdocument.in/reader036/viewer/2022062717/56649e305503460f94b206e9/html5/thumbnails/4.jpg)
Shifting PerspectivesShifting Perspectives
What happens if we substitute the word “learned” for the word “taught”?
What are the implications of “Where are learning outcomes learned and assessed”?
Emphasis on student demonstration, not topic-covering
Ability to do or apply supersedes knowing Responsibility for learning is shared Site of learning becomes less specific, and
boundaries become more fungible
![Page 5: Assessing Co-curricular Learning Jonathan Keiser Senior Officer, OASIS, The Higher Learning Commission Robert Mundhenk Assessment Consultant](https://reader036.vdocument.in/reader036/viewer/2022062717/56649e305503460f94b206e9/html5/thumbnails/5.jpg)
After After Learning ReconsideredLearning Reconsidered
“Learning” is not exclusively classroom-basedMany valued outcomes are not taught in the
classroomMany valued outcomes are the result of
processes outside the classroom “Learning” is a process based on three
interdependent student experiences: Understanding academic content and processes Student development Identity formation
Meaning is constructed from inquiry, interaction, conversation, and life experience
![Page 6: Assessing Co-curricular Learning Jonathan Keiser Senior Officer, OASIS, The Higher Learning Commission Robert Mundhenk Assessment Consultant](https://reader036.vdocument.in/reader036/viewer/2022062717/56649e305503460f94b206e9/html5/thumbnails/6.jpg)
After After Learning ReconsideredLearning Reconsidered
Responsibility for “learning” exists outside the classroom
Responsibility for “learning” doesn’t always take the same form; some entities on campus produce it, some facilitate it, some support it
Responsibility for assessing learning exists outside the classroom as well
![Page 7: Assessing Co-curricular Learning Jonathan Keiser Senior Officer, OASIS, The Higher Learning Commission Robert Mundhenk Assessment Consultant](https://reader036.vdocument.in/reader036/viewer/2022062717/56649e305503460f94b206e9/html5/thumbnails/7.jpg)
Assessment
Strategies, Methods, and Approaches for
UNDERSTANDING CONFIRMING IMPROVING
Student Learning
![Page 8: Assessing Co-curricular Learning Jonathan Keiser Senior Officer, OASIS, The Higher Learning Commission Robert Mundhenk Assessment Consultant](https://reader036.vdocument.in/reader036/viewer/2022062717/56649e305503460f94b206e9/html5/thumbnails/8.jpg)
How are your stated student learning outcomes appropriate to your mission, programs, students, and degrees?
Fundamental Question
![Page 9: Assessing Co-curricular Learning Jonathan Keiser Senior Officer, OASIS, The Higher Learning Commission Robert Mundhenk Assessment Consultant](https://reader036.vdocument.in/reader036/viewer/2022062717/56649e305503460f94b206e9/html5/thumbnails/9.jpg)
What evidence do you have that students achieve your stated learning outcomes?
Fundamental Question
![Page 10: Assessing Co-curricular Learning Jonathan Keiser Senior Officer, OASIS, The Higher Learning Commission Robert Mundhenk Assessment Consultant](https://reader036.vdocument.in/reader036/viewer/2022062717/56649e305503460f94b206e9/html5/thumbnails/10.jpg)
In what ways do you analyze and use evidence of student learning?
Fundamental Question
![Page 11: Assessing Co-curricular Learning Jonathan Keiser Senior Officer, OASIS, The Higher Learning Commission Robert Mundhenk Assessment Consultant](https://reader036.vdocument.in/reader036/viewer/2022062717/56649e305503460f94b206e9/html5/thumbnails/11.jpg)
How do you ensure shared responsibility for student learning & assessment of student learning?
Fundamental Question
![Page 12: Assessing Co-curricular Learning Jonathan Keiser Senior Officer, OASIS, The Higher Learning Commission Robert Mundhenk Assessment Consultant](https://reader036.vdocument.in/reader036/viewer/2022062717/56649e305503460f94b206e9/html5/thumbnails/12.jpg)
How do you evaluate and improve the effectiveness of your efforts to assess and improve student learning?
Fundamental Question
![Page 13: Assessing Co-curricular Learning Jonathan Keiser Senior Officer, OASIS, The Higher Learning Commission Robert Mundhenk Assessment Consultant](https://reader036.vdocument.in/reader036/viewer/2022062717/56649e305503460f94b206e9/html5/thumbnails/13.jpg)
In what ways do you inform the public about what students learn and how well?
Fundamental Question
![Page 14: Assessing Co-curricular Learning Jonathan Keiser Senior Officer, OASIS, The Higher Learning Commission Robert Mundhenk Assessment Consultant](https://reader036.vdocument.in/reader036/viewer/2022062717/56649e305503460f94b206e9/html5/thumbnails/14.jpg)
Some Post-Some Post-LRLR Examples of Examples of LearningLearning
Civic Responsibility AA: Service learning SA: Student government, voter registration,
student judicial boardsRespect for Other Cultures
AA: Language courses, Anthropology, Sociology SA: International experiences, culture days,
residence halls
![Page 15: Assessing Co-curricular Learning Jonathan Keiser Senior Officer, OASIS, The Higher Learning Commission Robert Mundhenk Assessment Consultant](https://reader036.vdocument.in/reader036/viewer/2022062717/56649e305503460f94b206e9/html5/thumbnails/15.jpg)
Some Core Areas Where Learning Some Core Areas Where Learning Can Occur in Non-classroom Can Occur in Non-classroom SettingsSettings
Critical ThinkingWorking with OthersGlobal Competence
CommunicationAdaptability
Preparation for Life-long LearningEthical Behavior
![Page 16: Assessing Co-curricular Learning Jonathan Keiser Senior Officer, OASIS, The Higher Learning Commission Robert Mundhenk Assessment Consultant](https://reader036.vdocument.in/reader036/viewer/2022062717/56649e305503460f94b206e9/html5/thumbnails/16.jpg)
Co-curricular OutcomesCo-curricular Outcomes
Need to be intentionalNeed to be plannedNeed to be part of the structure of a student’s
experienceNeed to be assessed Example: Missouri State University
◦ Living Learning Communities on designated floors of Residence Halls◦ Students contracts◦ Planned and intentional interactions with faculty
Example: Creighton University ◦ Faculty Partners Programs (recognition/rewards for faculty) ◦ Multicultural Affairs, Career Center, Office of Student Success
![Page 17: Assessing Co-curricular Learning Jonathan Keiser Senior Officer, OASIS, The Higher Learning Commission Robert Mundhenk Assessment Consultant](https://reader036.vdocument.in/reader036/viewer/2022062717/56649e305503460f94b206e9/html5/thumbnails/17.jpg)
Co-curricular AssessmentCo-curricular Assessment
Efficiency models: Focus on process How well is this office/service functioning? Focus on numbers:
Clients served Graduation rates Tutorial visits Attendance at activities Student/staff ratios
Example: Indiana East University ◦ Uses frequency data to identify areas with greatest potential
impact◦ Academic Advising, Student Support Services, and Athletics
![Page 18: Assessing Co-curricular Learning Jonathan Keiser Senior Officer, OASIS, The Higher Learning Commission Robert Mundhenk Assessment Consultant](https://reader036.vdocument.in/reader036/viewer/2022062717/56649e305503460f94b206e9/html5/thumbnails/18.jpg)
Co-curricular AssessmentCo-curricular Assessment
Effectiveness Models: Indirect◦Based on surveys and other indirect indicators,
like NSSE◦Often rely on student self-reporting◦Tend to skew positively on outcomes, if not
always on the processes that led to them Example: Saint Xavier University
◦ Gen Ed & Student Affairs host Campus Conversation Day◦ Observation: “62% of our students would find it ‘Very Difficult’
to ask instructors for help when struggling with course assignments.”
◦ Result: Biannual meetings to discuss first year data
![Page 19: Assessing Co-curricular Learning Jonathan Keiser Senior Officer, OASIS, The Higher Learning Commission Robert Mundhenk Assessment Consultant](https://reader036.vdocument.in/reader036/viewer/2022062717/56649e305503460f94b206e9/html5/thumbnails/19.jpg)
Co-curricular AssessmentCo-curricular Assessment
Effectiveness Models: Direct◦Focuses on student performance◦Can be based on observation or objective
measures◦Require carefully designed and consistent
measuring practicesExample: Drake University
◦ Student Life Lecture Series◦ Topics informed by Gen Ed Outcomes◦ Integrated Direct Assessment Measures◦ Forwarded to Assessment Committee for Analysis and
Communication
![Page 20: Assessing Co-curricular Learning Jonathan Keiser Senior Officer, OASIS, The Higher Learning Commission Robert Mundhenk Assessment Consultant](https://reader036.vdocument.in/reader036/viewer/2022062717/56649e305503460f94b206e9/html5/thumbnails/20.jpg)
How to Assess Co-curricular How to Assess Co-curricular LearningLearning
Apply external standards, like CAS
Use surveys and questionnaires
Develop direct measurement strategies
All of the aboveExample: Tulsa Community College
◦ RISE Program (Title IV TRiO grant)◦ Workshops & Activities based on Gen Ed Outcomes: ◦ Effective Communication, Critical Thinking, Technological Proficiency, and
Engaged Learning ◦ Integrated Direct Assessment Measures
![Page 21: Assessing Co-curricular Learning Jonathan Keiser Senior Officer, OASIS, The Higher Learning Commission Robert Mundhenk Assessment Consultant](https://reader036.vdocument.in/reader036/viewer/2022062717/56649e305503460f94b206e9/html5/thumbnails/21.jpg)
CAS StandardsCAS Standards
Standards for 35 functional areasThirteen component parts:
Mission Program Leadership Organization and management Human resources Financial resources Facilities, technology, and equipment Legal responsibilities Equity and access Campus and community relations Diversity Ethics Assessment and evaluation
Example: University of Akron (CAS Expert on Assessment Committee)
![Page 22: Assessing Co-curricular Learning Jonathan Keiser Senior Officer, OASIS, The Higher Learning Commission Robert Mundhenk Assessment Consultant](https://reader036.vdocument.in/reader036/viewer/2022062717/56649e305503460f94b206e9/html5/thumbnails/22.jpg)
CAS Standards: Areas of Focus for CAS Standards: Areas of Focus for Learning and DevelopmentLearning and Development
![Page 23: Assessing Co-curricular Learning Jonathan Keiser Senior Officer, OASIS, The Higher Learning Commission Robert Mundhenk Assessment Consultant](https://reader036.vdocument.in/reader036/viewer/2022062717/56649e305503460f94b206e9/html5/thumbnails/23.jpg)
FALDOsFALDOs
Frameworks for assessing learning and development outcomes
“Companion” to standardsExamples, strategies, and instruments for each
of the areas of focus Example: Briar Cliff University
◦ Focus Areas: Student Development and Athletics ◦ Student Development Themes:
Realistic Self-Appraisal Leadership Development Social Responsibility
◦ Athletics: Mission Statement ”Inspire Students to become Leaders in Life” guides programming and activities
![Page 24: Assessing Co-curricular Learning Jonathan Keiser Senior Officer, OASIS, The Higher Learning Commission Robert Mundhenk Assessment Consultant](https://reader036.vdocument.in/reader036/viewer/2022062717/56649e305503460f94b206e9/html5/thumbnails/24.jpg)
Questions about EvidenceQuestions about Evidence
Is it relevant to the area’s stated mission and function?
Does it measure what we want it to measure?
Does it deal in some way with outcomes?Is the information derived useful?Can the information be used to improve
either function or learning?
![Page 25: Assessing Co-curricular Learning Jonathan Keiser Senior Officer, OASIS, The Higher Learning Commission Robert Mundhenk Assessment Consultant](https://reader036.vdocument.in/reader036/viewer/2022062717/56649e305503460f94b206e9/html5/thumbnails/25.jpg)
Surveys and Questionnaires: Surveys and Questionnaires: Some Sample QuestionsSome Sample Questions
Did you accomplish what you hoped to accomplish in your meeting with your advisor?
How well did your experience at X prepare you for employment?
As a result of this First-Year program, do you feel better prepared for college?
Write a short essay in which you describe the ways in which your attitudes and values have changed as a result of your semester in Argentina.
![Page 26: Assessing Co-curricular Learning Jonathan Keiser Senior Officer, OASIS, The Higher Learning Commission Robert Mundhenk Assessment Consultant](https://reader036.vdocument.in/reader036/viewer/2022062717/56649e305503460f94b206e9/html5/thumbnails/26.jpg)
Surveys and Questionnaires: Surveys and Questionnaires: Indirect EvidenceIndirect Evidence
Traditional, indirect source of information on effectiveness
Limitations: Self-reporting Unvalidated opinion Response rates Opportunistic data Skewed samples
![Page 27: Assessing Co-curricular Learning Jonathan Keiser Senior Officer, OASIS, The Higher Learning Commission Robert Mundhenk Assessment Consultant](https://reader036.vdocument.in/reader036/viewer/2022062717/56649e305503460f94b206e9/html5/thumbnails/27.jpg)
Surveys and QuestionnairesSurveys and Questionnaires
Kinds: Satisfaction Reflective Post-experience experience (alumni and
employers)Value:
True “customer” response Can indicate areas for improvement and
ratification Provides data for planning and review
![Page 28: Assessing Co-curricular Learning Jonathan Keiser Senior Officer, OASIS, The Higher Learning Commission Robert Mundhenk Assessment Consultant](https://reader036.vdocument.in/reader036/viewer/2022062717/56649e305503460f94b206e9/html5/thumbnails/28.jpg)
Surveys and QuestionnairesSurveys and Questionnaires
Making them tools to assess learning: Use learning outcomes as basis for at least some
questions Validate by cross-referencing outcomes with
different populations (employers, alumni, graduate, current students)
Emphasize the learning outcomes in design and analysis
![Page 29: Assessing Co-curricular Learning Jonathan Keiser Senior Officer, OASIS, The Higher Learning Commission Robert Mundhenk Assessment Consultant](https://reader036.vdocument.in/reader036/viewer/2022062717/56649e305503460f94b206e9/html5/thumbnails/29.jpg)
Direct Effectiveness MeasuresDirect Effectiveness Measures
If Learning Reconsidered made the case for cross-campus responsibility for learning, then assessment of learning outcomes is also a cross-campus responsibility
Adaptation of practices and devices already in use in academic settings
Standardized Judgment-based
![Page 30: Assessing Co-curricular Learning Jonathan Keiser Senior Officer, OASIS, The Higher Learning Commission Robert Mundhenk Assessment Consultant](https://reader036.vdocument.in/reader036/viewer/2022062717/56649e305503460f94b206e9/html5/thumbnails/30.jpg)
What does a cross-campus responsibility What does a cross-campus responsibility for learning look like?for learning look like?
Culture is a shared, learned, symbolic system of values, beliefs and attitudes that shapes and influences perception and behavior.
Learned Shared
Mutually Constructed Internalized
Symbolic
![Page 31: Assessing Co-curricular Learning Jonathan Keiser Senior Officer, OASIS, The Higher Learning Commission Robert Mundhenk Assessment Consultant](https://reader036.vdocument.in/reader036/viewer/2022062717/56649e305503460f94b206e9/html5/thumbnails/31.jpg)
Developing Direct Measures of Developing Direct Measures of EffectivenessEffectiveness
Intentional Planning:◦Determine areas of responsibility: what
office/function might be a logical place to contribute to particular learning outcomes?
◦Plan the outcome-based purpose of the activity◦Design non-passive activities (watching a film
plus discussion; International Days as more than food, costumes, and dance)
◦Design outcome-focused opportunities for processing
![Page 32: Assessing Co-curricular Learning Jonathan Keiser Senior Officer, OASIS, The Higher Learning Commission Robert Mundhenk Assessment Consultant](https://reader036.vdocument.in/reader036/viewer/2022062717/56649e305503460f94b206e9/html5/thumbnails/32.jpg)
Being IntentionalBeing Intentional
Choose one of the outcomes below and determine a single co-curricular area that might have some responsibility for developing it. Name specific activities that might help develop the outcome and specify what their effect on the student should be.
Ethical behavior Teamwork Critical Thinking Communication
![Page 33: Assessing Co-curricular Learning Jonathan Keiser Senior Officer, OASIS, The Higher Learning Commission Robert Mundhenk Assessment Consultant](https://reader036.vdocument.in/reader036/viewer/2022062717/56649e305503460f94b206e9/html5/thumbnails/33.jpg)
Planning for the Long TermPlanning for the Long Term
Mapping:◦If the learning outcome is important, single
exposure isn’t enough◦How do first-year experiences differ from last-
year ones—or what difference is expected in student response?
◦How to assure student’s development of outcomes from first year to last?
![Page 34: Assessing Co-curricular Learning Jonathan Keiser Senior Officer, OASIS, The Higher Learning Commission Robert Mundhenk Assessment Consultant](https://reader036.vdocument.in/reader036/viewer/2022062717/56649e305503460f94b206e9/html5/thumbnails/34.jpg)
Regular AssessmentRegular Assessment
One-shot assessment produces haphazard results that are usually insufficient for planning improvement
Tie assessments to logical stages of development, based on an outcome map
Be consistent in approach to assessingOptions:
Standardized instruments Self-generated tools
![Page 35: Assessing Co-curricular Learning Jonathan Keiser Senior Officer, OASIS, The Higher Learning Commission Robert Mundhenk Assessment Consultant](https://reader036.vdocument.in/reader036/viewer/2022062717/56649e305503460f94b206e9/html5/thumbnails/35.jpg)
MappingMapping
Using the outcome you chose for the first exercise, plan a four-stage development process for that outcome, describing the experience the student will have, the developmental outcome desired, the “site”/area responsible for the outcome, and the assessment strategy(ies) to be used.
![Page 36: Assessing Co-curricular Learning Jonathan Keiser Senior Officer, OASIS, The Higher Learning Commission Robert Mundhenk Assessment Consultant](https://reader036.vdocument.in/reader036/viewer/2022062717/56649e305503460f94b206e9/html5/thumbnails/36.jpg)
Self-generated ToolsSelf-generated Tools
ObservationsExpert judgmentsStudent self-reflectionEmployer/supervisor judgmentsExample: Alverno College
◦ E-portfolios◦ Mastery > Grades◦ Focus on reflection and advising
![Page 37: Assessing Co-curricular Learning Jonathan Keiser Senior Officer, OASIS, The Higher Learning Commission Robert Mundhenk Assessment Consultant](https://reader036.vdocument.in/reader036/viewer/2022062717/56649e305503460f94b206e9/html5/thumbnails/37.jpg)
Using Self-Generated ToolsUsing Self-Generated Tools
Consistency across observers is crucial, so a rubric of some kind is essential
Holistic rubrics: broad judgments (Acceptable/Not Acceptable/Needs Improvement, numerical scores)
Descriptive rubrics: defined criteria and measures
![Page 38: Assessing Co-curricular Learning Jonathan Keiser Senior Officer, OASIS, The Higher Learning Commission Robert Mundhenk Assessment Consultant](https://reader036.vdocument.in/reader036/viewer/2022062717/56649e305503460f94b206e9/html5/thumbnails/38.jpg)
Descriptive RubricsDescriptive Rubrics
First, determine the aspects of student performance that would indicate he/she has achieved an outcome (e.g., one aspect of a “social justice” outcome might be “the student’s writing demonstrates sensitivity to issues of class and power”)
Second, define the specific things a student would have to do to show he/she has mastered that aspect (e.g., “Clear understanding of the ways in which economic status affects behavior.” (Criteria)
Finally, describe degrees of achievement for each criterion (Measures)
![Page 39: Assessing Co-curricular Learning Jonathan Keiser Senior Officer, OASIS, The Higher Learning Commission Robert Mundhenk Assessment Consultant](https://reader036.vdocument.in/reader036/viewer/2022062717/56649e305503460f94b206e9/html5/thumbnails/39.jpg)
Developing a RubricDeveloping a Rubric
Using the outcome and the functional area you chose earlier, develop a rubric to measure student achievement, defining one aspect of the outcome, one criterion for measuring that aspect, and a system (holistic, descriptive, whatever) for describing degrees of attainment
![Page 40: Assessing Co-curricular Learning Jonathan Keiser Senior Officer, OASIS, The Higher Learning Commission Robert Mundhenk Assessment Consultant](https://reader036.vdocument.in/reader036/viewer/2022062717/56649e305503460f94b206e9/html5/thumbnails/40.jpg)
Institutional AssessmentInstitutional Assessment
Assessments done across campus should ideally use the same rubrics or measuresExample: Graceland University
same rubric to measure Gen Ed Outcomes Co-curricular assessment committee Departmental Review and presentation to President
When using the same tools is not possible, it is essential that there be a way to extract information that is usable at the institutional level while still serving the needs of the functional area