assessment workshop college of san mateo february 2006
TRANSCRIPT
![Page 1: Assessment Workshop College of San Mateo February 2006](https://reader035.vdocument.in/reader035/viewer/2022070410/56649eb05503460f94bb5ee1/html5/thumbnails/1.jpg)
Assessment Workshop
College of San Mateo
February 2006
![Page 2: Assessment Workshop College of San Mateo February 2006](https://reader035.vdocument.in/reader035/viewer/2022070410/56649eb05503460f94bb5ee1/html5/thumbnails/2.jpg)
Program AssessmentProgram assessment is an on-going process
designed to monitor and improve student learning. Faculty:
• develop explicit statements of what students should learn.
• verify that the program is designed to foster this learning.
• collect empirical data that indicate student attainment.
• use these data to improve student learning.
![Page 3: Assessment Workshop College of San Mateo February 2006](https://reader035.vdocument.in/reader035/viewer/2022070410/56649eb05503460f94bb5ee1/html5/thumbnails/3.jpg)
Why so much emphasis on assessment?
• Accreditation Expectations
• Being Learning-Centered
• The Bottom Line
![Page 4: Assessment Workshop College of San Mateo February 2006](https://reader035.vdocument.in/reader035/viewer/2022070410/56649eb05503460f94bb5ee1/html5/thumbnails/4.jpg)
ACCJC Expectations
• For general education
• For assessment
![Page 5: Assessment Workshop College of San Mateo February 2006](https://reader035.vdocument.in/reader035/viewer/2022070410/56649eb05503460f94bb5ee1/html5/thumbnails/5.jpg)
Pop Quiz
1. ACCJC expects institutions to integrate learning outcomes into
a. programs
b. program review processes
c. syllabi
d. grading practices
e. all of the above
![Page 6: Assessment Workshop College of San Mateo February 2006](https://reader035.vdocument.in/reader035/viewer/2022070410/56649eb05503460f94bb5ee1/html5/thumbnails/6.jpg)
2. Who should control the assessment of student learning?a. Administrators
b. External consultants
c. Faculty
d. Institutional research professionals
![Page 7: Assessment Workshop College of San Mateo February 2006](https://reader035.vdocument.in/reader035/viewer/2022070410/56649eb05503460f94bb5ee1/html5/thumbnails/7.jpg)
Learning-Centered Institutions
• Academic program goals and curriculum
• How students learn
• Course structure and grading
• Pedagogy and course delivery
• Faculty instructional roles
• Assessment
• Campus support for learning
![Page 8: Assessment Workshop College of San Mateo February 2006](https://reader035.vdocument.in/reader035/viewer/2022070410/56649eb05503460f94bb5ee1/html5/thumbnails/8.jpg)
The Cohesive Curriculum
• Coherence
• Synthesizing Experiences
• Ongoing Practice of Learned Skills
• Increasing Sophistication and Application
![Page 9: Assessment Workshop College of San Mateo February 2006](https://reader035.vdocument.in/reader035/viewer/2022070410/56649eb05503460f94bb5ee1/html5/thumbnails/9.jpg)
Curriculum Alignment Matrix
• I = outcomes are introduced at the basic level
• D = students are given opportunities to practice, learn more about, and receive feedback to develop more sophistication
• M = students demonstrate mastery at a level appropriate for graduation
Is this a cohesive curriculum?
![Page 10: Assessment Workshop College of San Mateo February 2006](https://reader035.vdocument.in/reader035/viewer/2022070410/56649eb05503460f94bb5ee1/html5/thumbnails/10.jpg)
Course Planning
• Course Outcome
• Activity
• Assessment
![Page 11: Assessment Workshop College of San Mateo February 2006](https://reader035.vdocument.in/reader035/viewer/2022070410/56649eb05503460f94bb5ee1/html5/thumbnails/11.jpg)
Assessment Steps
1. Goals and outcomes
2. Alignment
3. A meaningful, manageable, sustainable assessment plan
4. Collect assessment data.
5. Close the loop.
6. Routinely examine the assessment process.
![Page 12: Assessment Workshop College of San Mateo February 2006](https://reader035.vdocument.in/reader035/viewer/2022070410/56649eb05503460f94bb5ee1/html5/thumbnails/12.jpg)
Never test the depth of the water with two feet.
![Page 13: Assessment Workshop College of San Mateo February 2006](https://reader035.vdocument.in/reader035/viewer/2022070410/56649eb05503460f94bb5ee1/html5/thumbnails/13.jpg)
Elements of an Assessment Plan
• Who?
• What?
• When? How often?
• Where?
• How?
![Page 14: Assessment Workshop College of San Mateo February 2006](https://reader035.vdocument.in/reader035/viewer/2022070410/56649eb05503460f94bb5ee1/html5/thumbnails/14.jpg)
Quotations from the Wise and Experienced
![Page 15: Assessment Workshop College of San Mateo February 2006](https://reader035.vdocument.in/reader035/viewer/2022070410/56649eb05503460f94bb5ee1/html5/thumbnails/15.jpg)
We don’t have to assess every outcome in every
student every year!
![Page 16: Assessment Workshop College of San Mateo February 2006](https://reader035.vdocument.in/reader035/viewer/2022070410/56649eb05503460f94bb5ee1/html5/thumbnails/16.jpg)
Vocabulary
• Direct vs. Indirect Assessment
• Quantitative vs. Qualitative Assessment
• Value-Added vs. Absolute Attainment
• Embedded Assessment
• Authentic Assessment
• Formative vs. Summative Assessment
![Page 17: Assessment Workshop College of San Mateo February 2006](https://reader035.vdocument.in/reader035/viewer/2022070410/56649eb05503460f94bb5ee1/html5/thumbnails/17.jpg)
Articulating Learning Outcomes:
• Knowledge
• Skills
• Values
![Page 18: Assessment Workshop College of San Mateo February 2006](https://reader035.vdocument.in/reader035/viewer/2022070410/56649eb05503460f94bb5ee1/html5/thumbnails/18.jpg)
Outcomes at Different Levels
• Course Session Level
• Course Level
• Program Level
• Institutional Level
![Page 19: Assessment Workshop College of San Mateo February 2006](https://reader035.vdocument.in/reader035/viewer/2022070410/56649eb05503460f94bb5ee1/html5/thumbnails/19.jpg)
Program Learning Outcomes:
• Focus on what students learn.
• Should be widely distributed.
• Should be known by all major stakeholders.
• Guide course and curriculum planning.
• Encourage students to be intentional learners.
• Focus assessment efforts.
![Page 20: Assessment Workshop College of San Mateo February 2006](https://reader035.vdocument.in/reader035/viewer/2022070410/56649eb05503460f94bb5ee1/html5/thumbnails/20.jpg)
Mission, Goals, and Outcomes
• Mission
• Goals
• Outcomes
![Page 21: Assessment Workshop College of San Mateo February 2006](https://reader035.vdocument.in/reader035/viewer/2022070410/56649eb05503460f94bb5ee1/html5/thumbnails/21.jpg)
Examples
Mission
Goals
Outcomes
![Page 22: Assessment Workshop College of San Mateo February 2006](https://reader035.vdocument.in/reader035/viewer/2022070410/56649eb05503460f94bb5ee1/html5/thumbnails/22.jpg)
Is each a mission, goal, or outcome?
![Page 23: Assessment Workshop College of San Mateo February 2006](https://reader035.vdocument.in/reader035/viewer/2022070410/56649eb05503460f94bb5ee1/html5/thumbnails/23.jpg)
Tips to Develop Program Goals and Outcomes
![Page 24: Assessment Workshop College of San Mateo February 2006](https://reader035.vdocument.in/reader035/viewer/2022070410/56649eb05503460f94bb5ee1/html5/thumbnails/24.jpg)
Possible Learning Goals
• Institution-Wide Goals
• Program-Specific Goals
![Page 25: Assessment Workshop College of San Mateo February 2006](https://reader035.vdocument.in/reader035/viewer/2022070410/56649eb05503460f94bb5ee1/html5/thumbnails/25.jpg)
Bloom’s Taxonomy
![Page 26: Assessment Workshop College of San Mateo February 2006](https://reader035.vdocument.in/reader035/viewer/2022070410/56649eb05503460f94bb5ee1/html5/thumbnails/26.jpg)
Effective Learning Outcomes
• Use active verbs to describe behaviors.
• Identify the expected depth of processing.
• Distinguish between absolute and value-added expectations.
![Page 27: Assessment Workshop College of San Mateo February 2006](https://reader035.vdocument.in/reader035/viewer/2022070410/56649eb05503460f94bb5ee1/html5/thumbnails/27.jpg)
Outcomes for Administrative and Academic Support Units
• Processes
• Learning Outcomes
• Satisfaction Indicators
![Page 28: Assessment Workshop College of San Mateo February 2006](https://reader035.vdocument.in/reader035/viewer/2022070410/56649eb05503460f94bb5ee1/html5/thumbnails/28.jpg)
Effective Outcomes
• Consistent with unit and campus mission
• Realistic
• Few in number
• Used by staff to set priorities and make decisions
![Page 29: Assessment Workshop College of San Mateo February 2006](https://reader035.vdocument.in/reader035/viewer/2022070410/56649eb05503460f94bb5ee1/html5/thumbnails/29.jpg)
Assessment Strategies for Administrative and Academic
Support Units
• Counts
• Client Satisfaction
• External Evaluations
• Learning Outcomes
![Page 30: Assessment Workshop College of San Mateo February 2006](https://reader035.vdocument.in/reader035/viewer/2022070410/56649eb05503460f94bb5ee1/html5/thumbnails/30.jpg)
Implementation Ideas, Insights, and Brainstorms
![Page 31: Assessment Workshop College of San Mateo February 2006](https://reader035.vdocument.in/reader035/viewer/2022070410/56649eb05503460f94bb5ee1/html5/thumbnails/31.jpg)
Assessment Techniques
• Direct
• Indirect
![Page 32: Assessment Workshop College of San Mateo February 2006](https://reader035.vdocument.in/reader035/viewer/2022070410/56649eb05503460f94bb5ee1/html5/thumbnails/32.jpg)
Properties of Good Assessment Techniques
• Valid
• Reliable
• Actionable
• Efficient and cost-effective
• Engaging to respondents
• Engaging to us
• Triangulation
![Page 33: Assessment Workshop College of San Mateo February 2006](https://reader035.vdocument.in/reader035/viewer/2022070410/56649eb05503460f94bb5ee1/html5/thumbnails/33.jpg)
Strategies for Direct Assessment
• Published Tests
• Locally-Developed Tests
• Embedded Assessment
• Portfolios
• Collective Portfolios
![Page 34: Assessment Workshop College of San Mateo February 2006](https://reader035.vdocument.in/reader035/viewer/2022070410/56649eb05503460f94bb5ee1/html5/thumbnails/34.jpg)
Implementation Ideas, Insights, and Brainstorms
![Page 35: Assessment Workshop College of San Mateo February 2006](https://reader035.vdocument.in/reader035/viewer/2022070410/56649eb05503460f94bb5ee1/html5/thumbnails/35.jpg)
Strategies for Indirect Assessment
• Surveys
• Interviews
• Focus Groups
![Page 36: Assessment Workshop College of San Mateo February 2006](https://reader035.vdocument.in/reader035/viewer/2022070410/56649eb05503460f94bb5ee1/html5/thumbnails/36.jpg)
Implementation Ideas, Insights, and Brainstorms
![Page 37: Assessment Workshop College of San Mateo February 2006](https://reader035.vdocument.in/reader035/viewer/2022070410/56649eb05503460f94bb5ee1/html5/thumbnails/37.jpg)
Developing and Applying Rubrics
• Holistic rubrics
• Analytic rubrics
![Page 38: Assessment Workshop College of San Mateo February 2006](https://reader035.vdocument.in/reader035/viewer/2022070410/56649eb05503460f94bb5ee1/html5/thumbnails/38.jpg)
Rubric Examples
![Page 39: Assessment Workshop College of San Mateo February 2006](https://reader035.vdocument.in/reader035/viewer/2022070410/56649eb05503460f94bb5ee1/html5/thumbnails/39.jpg)
Online Rubrics
![Page 40: Assessment Workshop College of San Mateo February 2006](https://reader035.vdocument.in/reader035/viewer/2022070410/56649eb05503460f94bb5ee1/html5/thumbnails/40.jpg)
Rubric Strengths
• Complex products or behaviors can be examined efficiently.
• Developing a rubric helps to precisely define faculty expectations.
• Well-trained reviewers apply the same criteria and standards.
• Rubrics are criterion-referenced, rather than norm-referenced.
• Ratings can be done by faculty or others.
![Page 41: Assessment Workshop College of San Mateo February 2006](https://reader035.vdocument.in/reader035/viewer/2022070410/56649eb05503460f94bb5ee1/html5/thumbnails/41.jpg)
Rubrics can be useful for grading, as well as assessment.
• Points for grading vary among faculty
• Categories are used for assessment
• Opportunity to provide formative feedback to students
![Page 42: Assessment Workshop College of San Mateo February 2006](https://reader035.vdocument.in/reader035/viewer/2022070410/56649eb05503460f94bb5ee1/html5/thumbnails/42.jpg)
Using Rubrics in Courses
1. Hand out rubric with assignment.
2. Use rubric for grading.
3. Develop rubric with students.
4. Students apply rubric to examples.
5. Peer feedback using rubric.
6. Self-assessment using rubric.
![Page 43: Assessment Workshop College of San Mateo February 2006](https://reader035.vdocument.in/reader035/viewer/2022070410/56649eb05503460f94bb5ee1/html5/thumbnails/43.jpg)
Generic Rubric
Check for inter-rater reliability to see if it works.
![Page 44: Assessment Workshop College of San Mateo February 2006](https://reader035.vdocument.in/reader035/viewer/2022070410/56649eb05503460f94bb5ee1/html5/thumbnails/44.jpg)
Creating a Rubric
• Adapt an existing rubric
• Analytic approach
• Expert-systems approach
![Page 45: Assessment Workshop College of San Mateo February 2006](https://reader035.vdocument.in/reader035/viewer/2022070410/56649eb05503460f94bb5ee1/html5/thumbnails/45.jpg)
Managing Group Readings
• One reader/document
• Two independent readers/document
• Paired readers
![Page 46: Assessment Workshop College of San Mateo February 2006](https://reader035.vdocument.in/reader035/viewer/2022070410/56649eb05503460f94bb5ee1/html5/thumbnails/46.jpg)
Before Inviting Colleagues
• Develop and pilot test the rubric.
• Select exemplars.
• Develop a recording system.
• Consider pre-programming a spreadsheet for data collection.
![Page 47: Assessment Workshop College of San Mateo February 2006](https://reader035.vdocument.in/reader035/viewer/2022070410/56649eb05503460f94bb5ee1/html5/thumbnails/47.jpg)
Rubric Orientation and Calibration
![Page 48: Assessment Workshop College of San Mateo February 2006](https://reader035.vdocument.in/reader035/viewer/2022070410/56649eb05503460f94bb5ee1/html5/thumbnails/48.jpg)
Implementation Ideas, Insights, and Brainstorms