navigating the process of student learning outcomes: development, evaluation, and improvement
DESCRIPTION
Navigating the Process of Student Learning Outcomes: Development, Evaluation, and Improvement. Shannon M. Sexton, Julia M. Williams, & Timothy Chow Rose- Hulman Institute of Technology. Introduction & Workshop Goals. Introduction Goals - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Navigating the Process of Student Learning Outcomes: Development, Evaluation, and Improvement](https://reader036.vdocument.in/reader036/viewer/2022081517/56816147550346895dd0c2aa/html5/thumbnails/1.jpg)
NAVIGATING THE PROCESS OF STUDENT LEARNING OUTCOMES: DEVELOPMENT, EVALUATION, AND IMPROVEMENT
Shannon M. Sexton, Julia M. Williams, & Timothy ChowRose-Hulman Institute of Technology
![Page 2: Navigating the Process of Student Learning Outcomes: Development, Evaluation, and Improvement](https://reader036.vdocument.in/reader036/viewer/2022081517/56816147550346895dd0c2aa/html5/thumbnails/2.jpg)
Introduction & Workshop Goals
RHIT/IRPA - NASPA 2009
Introduction Goals
1. Discuss the process of developing institute student learning outcomes as well as common difficulties.
2. Assist participants in developing their own learning outcomes3. Discuss evidence that can be used to evaluate student learning
outcomes achievement.4. Assist participants in identifying articles of evidence for their
learning outcomes.5. Discuss using an e-portfolio as a means of collecting and rating
evidence.6. Allow participants to rate sample evidence using a sample rating
rubric.7. Discuss how to use the rating results to close the loop and
improve student learning.
![Page 3: Navigating the Process of Student Learning Outcomes: Development, Evaluation, and Improvement](https://reader036.vdocument.in/reader036/viewer/2022081517/56816147550346895dd0c2aa/html5/thumbnails/3.jpg)
Group Activity #1
RHIT/IRPA - NASPA 2009
Define Teamwork Include characteristics of teamwork Include traits of those characteristics Time allotted 10 minutes
![Page 4: Navigating the Process of Student Learning Outcomes: Development, Evaluation, and Improvement](https://reader036.vdocument.in/reader036/viewer/2022081517/56816147550346895dd0c2aa/html5/thumbnails/4.jpg)
About Rose-Hulman Terre Haute, Indiana 1800+ undergraduate
students B.S. degrees in
engineering, science, and mathematics
80%+ engineering students
Accredited through ABET and North Central
RHIT/IRPA - NASPA 2009
![Page 5: Navigating the Process of Student Learning Outcomes: Development, Evaluation, and Improvement](https://reader036.vdocument.in/reader036/viewer/2022081517/56816147550346895dd0c2aa/html5/thumbnails/5.jpg)
CASO
RHIT/IRPA - NASPA 2009
Commission on the Assessment of Student Outcomes
Faculty committee with 1 representative from each academic department
Regular meetings throughout the academic year
Tasked with developing Institute-wide student learning outcomes and a way to measure them
![Page 6: Navigating the Process of Student Learning Outcomes: Development, Evaluation, and Improvement](https://reader036.vdocument.in/reader036/viewer/2022081517/56816147550346895dd0c2aa/html5/thumbnails/6.jpg)
Historic Timeline
RHIT/IRPA - NASPA 2009
1996 – 1998 1999 – 2000 2001 – 2006 2007 – present
![Page 7: Navigating the Process of Student Learning Outcomes: Development, Evaluation, and Improvement](https://reader036.vdocument.in/reader036/viewer/2022081517/56816147550346895dd0c2aa/html5/thumbnails/7.jpg)
Removing the Jargon
RHIT/IRPA - NASPA 2009
Domains = Categories of skills Learning Outcomes = Definition of the
skills students should have Performance Criteria = Characteristics of the learning outcome
Rubrics = Use of a scale to identify traits in a document
![Page 8: Navigating the Process of Student Learning Outcomes: Development, Evaluation, and Improvement](https://reader036.vdocument.in/reader036/viewer/2022081517/56816147550346895dd0c2aa/html5/thumbnails/8.jpg)
Domains and Performance Criteria
8
Domain Student Learning Outcome
Performance Criteria
Technical Skills Outcomes developed by each department.
Levels A – C indicate the level of performance achieved by each student.
Professional Skills
1. Leadership2. Teamwork3. Communication
Levels A – C indicate the level of performance achieved by each student.
World Citizenship 4. Cultural & Global Awareness
5. Ethics6. Service
Levels A – C indicate the level of performance achieved by each student.
RHIT/IRPA - NASPA 2009
![Page 9: Navigating the Process of Student Learning Outcomes: Development, Evaluation, and Improvement](https://reader036.vdocument.in/reader036/viewer/2022081517/56816147550346895dd0c2aa/html5/thumbnails/9.jpg)
Developing Outcomes
RHIT/IRPA - NASPA 2009
Importance of preliminary research Teams of faculty developers Developing in steps & gaining institute
buy-in Importance of measurability
![Page 10: Navigating the Process of Student Learning Outcomes: Development, Evaluation, and Improvement](https://reader036.vdocument.in/reader036/viewer/2022081517/56816147550346895dd0c2aa/html5/thumbnails/10.jpg)
Writing Outcomes – Activity #2
RHIT/IRPA - NASPA 2009
Revise your teamwork definition into a student learning outcome with rubrics
Time allotted 15-20 minutes
![Page 11: Navigating the Process of Student Learning Outcomes: Development, Evaluation, and Improvement](https://reader036.vdocument.in/reader036/viewer/2022081517/56816147550346895dd0c2aa/html5/thumbnails/11.jpg)
10 minutes to stretch your legs
Take a Break
RHIT/IRPA - NASPA 2009
![Page 12: Navigating the Process of Student Learning Outcomes: Development, Evaluation, and Improvement](https://reader036.vdocument.in/reader036/viewer/2022081517/56816147550346895dd0c2aa/html5/thumbnails/12.jpg)
Evidence of Outcomes
RHIT/IRPA - NASPA 2009
Choosing appropriate evidence New vs. Current Importance of Rubric
Collecting evidence How to physically collect & store evidence
Course Mapping Tracking where submissions are coming
from Accountability
![Page 13: Navigating the Process of Student Learning Outcomes: Development, Evaluation, and Improvement](https://reader036.vdocument.in/reader036/viewer/2022081517/56816147550346895dd0c2aa/html5/thumbnails/13.jpg)
Criterion Description and Rubric Teamwork B1: Demonstrate how you reached a
decision as a team.
Primary Traits: A passing submission for this criterion must:
1. Describe the team goal.2. Provide a description of a specific team decision and describe the following:
- the process of making the decision- how multiple team members contributed to the outcome- how team members’ ideas were critically evaluated
3. Show that the ultimate outcome was consistent with the team’s decision-making process.Potential documents: Documents appropriate for this criterion include (but are not limited to): Memo or reflective statement on team process from a lab group design team, debate team, service project.
Additional information:
13
RHIT/IRPA - NASPA 2009
![Page 14: Navigating the Process of Student Learning Outcomes: Development, Evaluation, and Improvement](https://reader036.vdocument.in/reader036/viewer/2022081517/56816147550346895dd0c2aa/html5/thumbnails/14.jpg)
Course Mapping14
Course mapping
RHIT/IRPA - NASPA 2009
![Page 15: Navigating the Process of Student Learning Outcomes: Development, Evaluation, and Improvement](https://reader036.vdocument.in/reader036/viewer/2022081517/56816147550346895dd0c2aa/html5/thumbnails/15.jpg)
Identifying Evidence – Activity #3 Where do you already have existing
assignments that could be used as articles of evidence for the Teamwork outcome? Group Brainstorming Activity
15
RHIT/IRPA - NASPA 2009
![Page 16: Navigating the Process of Student Learning Outcomes: Development, Evaluation, and Improvement](https://reader036.vdocument.in/reader036/viewer/2022081517/56816147550346895dd0c2aa/html5/thumbnails/16.jpg)
Domains and Performance Criteria
16
Performance CriteriaLevel C – associated with awareness, application, or evaluation.Level B – associated with application.Level A – associated with knowledge or comprehension.
A-Level Criteria
B-Level Criteria
C-Level Criteria
RHIT/IRPA - NASPA 2009
![Page 17: Navigating the Process of Student Learning Outcomes: Development, Evaluation, and Improvement](https://reader036.vdocument.in/reader036/viewer/2022081517/56816147550346895dd0c2aa/html5/thumbnails/17.jpg)
Collecting Evidence
RHIT/IRPA - NASPA 2009
Use of an E-portfolio system Faculty input Student input Assessment staff input
![Page 18: Navigating the Process of Student Learning Outcomes: Development, Evaluation, and Improvement](https://reader036.vdocument.in/reader036/viewer/2022081517/56816147550346895dd0c2aa/html5/thumbnails/18.jpg)
Rating Evidence
RHIT/IRPA - NASPA 2009
Rating preparation Administrative side of rating
Paying faculty volunteers, tech support, number of volunteers, etc.
Process of rating days
![Page 19: Navigating the Process of Student Learning Outcomes: Development, Evaluation, and Improvement](https://reader036.vdocument.in/reader036/viewer/2022081517/56816147550346895dd0c2aa/html5/thumbnails/19.jpg)
Portfolio Rating Methodology Evaluation rubric, A, B, C Inter-rater reliability: initial, subsequent Document rating: Pass/Fail/Exemplary Wholistic rating, NOT grading
RHIT/IRPA - NASPA 2009
![Page 20: Navigating the Process of Student Learning Outcomes: Development, Evaluation, and Improvement](https://reader036.vdocument.in/reader036/viewer/2022081517/56816147550346895dd0c2aa/html5/thumbnails/20.jpg)
RHIT/IRPA - NASPA 2009
![Page 21: Navigating the Process of Student Learning Outcomes: Development, Evaluation, and Improvement](https://reader036.vdocument.in/reader036/viewer/2022081517/56816147550346895dd0c2aa/html5/thumbnails/21.jpg)
RHIT/IRPA - NASPA 2009
![Page 22: Navigating the Process of Student Learning Outcomes: Development, Evaluation, and Improvement](https://reader036.vdocument.in/reader036/viewer/2022081517/56816147550346895dd0c2aa/html5/thumbnails/22.jpg)
Rating Documents – Activity #4
RHIT/IRPA - NASPA 2009
Rate 2-3 teamwork documents using the provided rubric.
15 - 20 minutes
![Page 23: Navigating the Process of Student Learning Outcomes: Development, Evaluation, and Improvement](https://reader036.vdocument.in/reader036/viewer/2022081517/56816147550346895dd0c2aa/html5/thumbnails/23.jpg)
Using Rating Results
RHIT/IRPA - NASPA 2009
Identifying constituents How to communicate results to institute What to do with the results
![Page 24: Navigating the Process of Student Learning Outcomes: Development, Evaluation, and Improvement](https://reader036.vdocument.in/reader036/viewer/2022081517/56816147550346895dd0c2aa/html5/thumbnails/24.jpg)
Example Result
RHIT/IRPA - NASPA 2009
2005 2006 20070%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
36%
63%71%
52%
62%
73%65%
74%83%
Historical Portfolio Rating Results
Teamwork Communication Ethics
Mee
ts S
tand
ard
![Page 25: Navigating the Process of Student Learning Outcomes: Development, Evaluation, and Improvement](https://reader036.vdocument.in/reader036/viewer/2022081517/56816147550346895dd0c2aa/html5/thumbnails/25.jpg)
Please contact us for more information after the conference.
RHIT/IRPA - NASPA 2009