anne fulkerson, ph.d. institutional research
DESCRIPTION
Developing an Assessment Plan Owens Community College Assessment Day Workshops November 13-14, 2009 . Anne Fulkerson, Ph.D. Institutional Research. Agenda. What is assessment? Why assess? Types of assessment Basic steps in the assessment planning process Identifying outcomes - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Anne Fulkerson, Ph.D. Institutional Research](https://reader036.vdocument.in/reader036/viewer/2022062521/56816714550346895ddb7fc9/html5/thumbnails/1.jpg)
Developing an Assessment Plan
Owens Community College Assessment Day WorkshopsNovember 13-14, 2009
Anne Fulkerson, Ph.D.Institutional Research
![Page 2: Anne Fulkerson, Ph.D. Institutional Research](https://reader036.vdocument.in/reader036/viewer/2022062521/56816714550346895ddb7fc9/html5/thumbnails/2.jpg)
Agenda
What is assessment? Why assess? Types of assessment Basic steps in the assessment planning process
Identifying outcomes Defining measures Evaluating measures Other important considerations
OCC assessment plan template & rubric Sample assessment plan How IR can help
![Page 3: Anne Fulkerson, Ph.D. Institutional Research](https://reader036.vdocument.in/reader036/viewer/2022062521/56816714550346895ddb7fc9/html5/thumbnails/3.jpg)
What is Assessment?
“Assessment is the ongoing process of:
Establishing clear, measurable expected outcomes of student learning.
Ensuring that students have sufficient opportunities to achieve those outcomes.
Systematically gathering, analyzing, and interpreting evidence to determine how well student learning matches our expectations.
Using the resulting information to understand and improve student learning.”
Suskie (2004), pg. 3
![Page 4: Anne Fulkerson, Ph.D. Institutional Research](https://reader036.vdocument.in/reader036/viewer/2022062521/56816714550346895ddb7fc9/html5/thumbnails/4.jpg)
Why Assess?
Demonstrate effectiveness Improve student learning
Measure Learning
Adapted from Suskie (2004) and Maki (2004)
Provide Learning Opportunities
Establish Learning Outcomes
Implement Change
![Page 5: Anne Fulkerson, Ph.D. Institutional Research](https://reader036.vdocument.in/reader036/viewer/2022062521/56816714550346895ddb7fc9/html5/thumbnails/5.jpg)
Types of Assessment
Formative—assessment that takes place while learning is in progress (or while a new program is being developed) in order to provide feedback for improvement
Summative—assessment that takes place at the end of a course or program to document that learning has occurred or to make judgments about the efficacy of a program
![Page 6: Anne Fulkerson, Ph.D. Institutional Research](https://reader036.vdocument.in/reader036/viewer/2022062521/56816714550346895ddb7fc9/html5/thumbnails/6.jpg)
Establish Mission Statement
![Page 7: Anne Fulkerson, Ph.D. Institutional Research](https://reader036.vdocument.in/reader036/viewer/2022062521/56816714550346895ddb7fc9/html5/thumbnails/7.jpg)
Identify Student Learning Outcomes
Student Learning Outcomes: Specifiable activities, products, skills, abilities, behaviors, attitudes, or pieces of knowledge that students attain as a result of their involvement in a particular set of educational experiences.
Represent the most important competencies that all students should possess
Do not reflect ALL learning that might occur Often aligned with accrediting body standards No magic number, but keep them manageable
![Page 8: Anne Fulkerson, Ph.D. Institutional Research](https://reader036.vdocument.in/reader036/viewer/2022062521/56816714550346895ddb7fc9/html5/thumbnails/8.jpg)
Expressing Student Learning Outcomes
Too vague:Students will demonstrate information literacy skills
Too specific:Students will be able to use institutional online services toretrieve information
Better:Students will locate information and evaluate it criticallyfor its validity and appropriateness
![Page 9: Anne Fulkerson, Ph.D. Institutional Research](https://reader036.vdocument.in/reader036/viewer/2022062521/56816714550346895ddb7fc9/html5/thumbnails/9.jpg)
Expressing Student Learning Outcomes
Fuzzy Terms: Action Words:
KnowUnderstandBecome aware ofAppreciateThink criticallyDemonstrate knowledgeLearnWrite proficiently
RememberIdentifyPerformCreateDefineSummarizeExplainDiscussDescribeSolveFind/LocateAnalyze
EvaluateApplyDebate/ArgueSynthesizeIntegrateResearchChooseConstructDesignDevelopOrganizeUse
![Page 10: Anne Fulkerson, Ph.D. Institutional Research](https://reader036.vdocument.in/reader036/viewer/2022062521/56816714550346895ddb7fc9/html5/thumbnails/10.jpg)
Bloom’s Taxonomy (Revised)
Based on an adaptation of Anderson & Krathwohl (2001)http://uwf.edu/cutla/assessstudent.cfm
Thin
king
Ski
lls
![Page 11: Anne Fulkerson, Ph.D. Institutional Research](https://reader036.vdocument.in/reader036/viewer/2022062521/56816714550346895ddb7fc9/html5/thumbnails/11.jpg)
Not Measurable:
Recognize a need for lifelong learning and plan for personal andprofessional growth
Demonstrate an historical knowledge of the symphonic, string orchestra, and chamberensemble repertoire
Sample Student Learning Outcomes
Measurable:
Describe and adopt a plan for ongoing professionaldevelopment and lifelong learning
Describe the historical development of the symphonic, string orchestra, and chamberensemble repertoire
http://uwf.edu/cutla/assessstudent.cfm
![Page 12: Anne Fulkerson, Ph.D. Institutional Research](https://reader036.vdocument.in/reader036/viewer/2022062521/56816714550346895ddb7fc9/html5/thumbnails/12.jpg)
Identify Opportunities for Students to Achieve Each Learning Outcome
Course / Activity / ExperienceLearning Outcome
1.
2.
3.
4.
5.
![Page 13: Anne Fulkerson, Ph.D. Institutional Research](https://reader036.vdocument.in/reader036/viewer/2022062521/56816714550346895ddb7fc9/html5/thumbnails/13.jpg)
Define Measures for Each Outcome
Types of Measures:
Quantitative—numeric (e.g., test scores, placement rates, GPA, structured surveys)
Qualitative—described by words rather than numbers (e.g., interviews, focus groups, observations, rubrics)
Direct—require students to display knowledge or skill (e.g., tests, performances, assignments)
Indirect—second-hand evidence (e.g., surveys, course evaluations, journal reflections)
![Page 14: Anne Fulkerson, Ph.D. Institutional Research](https://reader036.vdocument.in/reader036/viewer/2022062521/56816714550346895ddb7fc9/html5/thumbnails/14.jpg)
Word of Caution
Don’t measure everything, just because you can Pick the best measures Keep it manageable
![Page 15: Anne Fulkerson, Ph.D. Institutional Research](https://reader036.vdocument.in/reader036/viewer/2022062521/56816714550346895ddb7fc9/html5/thumbnails/15.jpg)
Determine How Measures will be Evaluated Criterion-based benchmarks—compares
student performance with a pre-established standard. Norm-referenced benchmarks—compares
student performance with a standardized norm or a group of peers.
Value-added approach—compares student performance at two points in time to see if they have improved.
Longitudinal/historical approach—compares current students with prior students.
![Page 16: Anne Fulkerson, Ph.D. Institutional Research](https://reader036.vdocument.in/reader036/viewer/2022062521/56816714550346895ddb7fc9/html5/thumbnails/16.jpg)
Other Things to Consider
What resources are needed?
Who’s responsible for collecting and analyzing the data?
Build support through participation in the planning process
Assessment plans are perpetual drafts
![Page 17: Anne Fulkerson, Ph.D. Institutional Research](https://reader036.vdocument.in/reader036/viewer/2022062521/56816714550346895ddb7fc9/html5/thumbnails/17.jpg)
OCC Assessment Plan Templatehttps://intranet.owens.edu/committees/outcomes/index.html
Program Mission: Program Vision:
Program Level Student Learning Outcomes Program Student Learning Outcomes # 1: Where taught? Where
Where measured?
Direct measure(s)
Indirect measure(s)
Resources needed
Person(s) responsible
Schedule Benchmark(s)
1. 2.
1. 2.
1. 2.
1. 2.
Program SLO # 2. Where taught?
Where measured?
Direct measure(s)
Indirect measurer(s)
Resources needed
Person(s) responsible
Schedule Benchmark(s)
1. 2.
1. 2.
1. 2.
1. 2.
Program SLO # 3 Where taught? Where
measured? Direct measure(s)
Indirect measurer(s)
Resources needed
Person(s) responsible
Schedule Benchmark(s)
1. 2.
1. 2.
1. 2.
1. 2.
Program SLO # 4 Where taught? Where
measured? Direct measurer(s)
Indirect measurer(s)
Resources needed
Person(s) responsible
Schedule Benchmarks(s)
1. 2.
1. 2.
1. 2.
1. 2.
Program SLO # 5 Where taught? Where
measured? Direct measurer(s)
Indirect measurer(s)
Resources needed
Person(s) responsible
Schedule Benchmark(s)
1. 2.
1. 2.
1. 2.
1. 2.
![Page 18: Anne Fulkerson, Ph.D. Institutional Research](https://reader036.vdocument.in/reader036/viewer/2022062521/56816714550346895ddb7fc9/html5/thumbnails/18.jpg)
OCC Assessment Plan & Report Rubrichttps://intranet.owens.edu/committees/outcomes/index.html
Elements (1) Developing Established Exemplary Program goals and intended student learning outcomes (Appropriateness)
Lack of or incomplete inclusion of some established criteria.
Describe how the goals & SLO align with College &/or School mission & vision. Describe how the goals & SLO link to accreditation standards. if app Describe how the goals & SLO link to industry and/or academic standards.
Provides evidence supporting the statements in “Established” category.
Evidence is appropriate for the measurement of student learning (Evidence)
Lack of or incomplete inclusion of some established criteria.
Evidence that the measurement instrument is aligned with goals and/or SLO. Benchmarks in place and evaluated regularly Artifacts when utilized are evaluated with rubrics, when appropriate. At least one direct & one indirect measure as appropriate.
Normed evidence Provide information on validity & reliability of instrumentation. Rubrics are constructed and/or agreed upon by most stakeholders. Multiple direct & indirect measures.
Individuals responsible for delivering the program engage in systematic analysis of evidence. (Analysis of Evidence & Shared Responsibility)
Lack of or incomplete inclusion of some established criteria.
Describe how the data was analyzed. Provide evidence that appropriate stakeholders were involved in the analysis of data. Describe how the data was used to make decisions about program improvement. Describe action steps needed to improve data collection and/or program. Provide a summary of the analysis with references to data.
Provide evidence that the previous plan of action was implemented. Provide evidence that the previous action plan was assessed. Provide results of the implementation of the action plan.
Results are shared with essential stakeholders
(Shared Responsibility)
Lack of or incomplete inclusion of some established criteria.
Evidence that the report was shared with all faculty and advisory committee members where applicable.
Evidence that the report was shared with additional stakeholder groups.
![Page 19: Anne Fulkerson, Ph.D. Institutional Research](https://reader036.vdocument.in/reader036/viewer/2022062521/56816714550346895ddb7fc9/html5/thumbnails/19.jpg)
Sample Assessment Plans
Owens— https://www.owens.edu/portrait/index.html
College of Charleston, SC— http://spinner.cofc.edu/~oap/docs.html
Supplemental Instruction
![Page 20: Anne Fulkerson, Ph.D. Institutional Research](https://reader036.vdocument.in/reader036/viewer/2022062521/56816714550346895ddb7fc9/html5/thumbnails/20.jpg)
How IR Can Help
Data support and analysis Assessment consulting
Survey development Rubric development Identifying appropriate metrics Identifying existing data sources
![Page 21: Anne Fulkerson, Ph.D. Institutional Research](https://reader036.vdocument.in/reader036/viewer/2022062521/56816714550346895ddb7fc9/html5/thumbnails/21.jpg)
References
Maki, P. (2004). Assessing for Learning: Building a Sustainable Commitment Across the Institution. Stylus Publishing: Herndon, VA.
Palomba, C. & Banta, T.W. (1999). Assessment Essentials. Planning, Implementing, and Improving Assessment in Higher Education. John Wiley & Sons: San Francisco, CA.
Suskie, L. (2004). Assessing Student Learning: A Common Sense Guide. Jossey-Bass: San Francisco, CA.
Owens Community College, Student Learning Assessment Committee (2009). Building your SLAC assessment plan: Instructions for the SLAC assessment plan template version #1: intranet.owens.edu/committees/outcomes/index.html.
University of West Florida, Center for University Teaching, Learning, & Assessment. http://uwf.edu/cutla/Assessres.cfm.