the academic program review bridging standards 7 and 14 middle states annual conference december 10,...
TRANSCRIPT
![Page 1: The Academic Program Review Bridging Standards 7 and 14 Middle States Annual Conference December 10, 2010](https://reader036.vdocument.in/reader036/viewer/2022070305/5514b0f1550346b2598b62b6/html5/thumbnails/1.jpg)
The Academic Program Review
Bridging Standards 7 and 14
Middle State’s Annual Conference
December 10, 2010
![Page 2: The Academic Program Review Bridging Standards 7 and 14 Middle States Annual Conference December 10, 2010](https://reader036.vdocument.in/reader036/viewer/2022070305/5514b0f1550346b2598b62b6/html5/thumbnails/2.jpg)
Presenters
Mr. H. Leon Hill
Director of Institutional Research
Dr. Joan E. Brookshire
Associate Vice President of Academic Affairs
![Page 3: The Academic Program Review Bridging Standards 7 and 14 Middle States Annual Conference December 10, 2010](https://reader036.vdocument.in/reader036/viewer/2022070305/5514b0f1550346b2598b62b6/html5/thumbnails/3.jpg)
Overview
• Framework to Address the APRs
• Structure/Challenges/Approach
• Examples of Metrics
• Current Action Plan
• Integration of “End User” Technology
• Next Steps
• Benefits of Our Approach
• Questions
![Page 4: The Academic Program Review Bridging Standards 7 and 14 Middle States Annual Conference December 10, 2010](https://reader036.vdocument.in/reader036/viewer/2022070305/5514b0f1550346b2598b62b6/html5/thumbnails/4.jpg)
Assessment Cycle-2005
![Page 5: The Academic Program Review Bridging Standards 7 and 14 Middle States Annual Conference December 10, 2010](https://reader036.vdocument.in/reader036/viewer/2022070305/5514b0f1550346b2598b62b6/html5/thumbnails/5.jpg)
What we had to build on
• Strong focus on programs.
• State mandated 5-year academic program review in need of revision.
• Institutional Effectiveness Model (IEM) with performance indicators benchmarked through State and National data bases.
![Page 6: The Academic Program Review Bridging Standards 7 and 14 Middle States Annual Conference December 10, 2010](https://reader036.vdocument.in/reader036/viewer/2022070305/5514b0f1550346b2598b62b6/html5/thumbnails/6.jpg)
Mission
Strategic Initiative:
Access & Success
Institutional Effectiveness
![Page 7: The Academic Program Review Bridging Standards 7 and 14 Middle States Annual Conference December 10, 2010](https://reader036.vdocument.in/reader036/viewer/2022070305/5514b0f1550346b2598b62b6/html5/thumbnails/7.jpg)
IEM
• Needed a way to assess how the College was performing on key metrics in relation to prior. years/semesters and compared to other institutions.
• Historical/Trend data
• Benchmark data– Pennsylvania & National Peers
![Page 8: The Academic Program Review Bridging Standards 7 and 14 Middle States Annual Conference December 10, 2010](https://reader036.vdocument.in/reader036/viewer/2022070305/5514b0f1550346b2598b62b6/html5/thumbnails/8.jpg)
• Institutional Effectiveness Model
![Page 9: The Academic Program Review Bridging Standards 7 and 14 Middle States Annual Conference December 10, 2010](https://reader036.vdocument.in/reader036/viewer/2022070305/5514b0f1550346b2598b62b6/html5/thumbnails/9.jpg)
Where we started
• Restructured the Academic Program Review process
• Incorporated the use of technology
![Page 10: The Academic Program Review Bridging Standards 7 and 14 Middle States Annual Conference December 10, 2010](https://reader036.vdocument.in/reader036/viewer/2022070305/5514b0f1550346b2598b62b6/html5/thumbnails/10.jpg)
Goal of the restructuring
• Measure student performance as evidence by results of assessment of student learning outcomes.
• Measure program performance as evidenced by comparison of program performance to overall college performance on specific key indicator (current and aspirational).
![Page 11: The Academic Program Review Bridging Standards 7 and 14 Middle States Annual Conference December 10, 2010](https://reader036.vdocument.in/reader036/viewer/2022070305/5514b0f1550346b2598b62b6/html5/thumbnails/11.jpg)
Challenges
• Usual issues with assessment in general.
• Faculty had little knowledge of the College’s performance indicators.
• Organizational separation of assessment of institutional and student learning outcomes.
![Page 12: The Academic Program Review Bridging Standards 7 and 14 Middle States Annual Conference December 10, 2010](https://reader036.vdocument.in/reader036/viewer/2022070305/5514b0f1550346b2598b62b6/html5/thumbnails/12.jpg)
Approach
Began by building it backwards from the IEM by mapping out specific core indicators to program data, making additions where needed.
![Page 13: The Academic Program Review Bridging Standards 7 and 14 Middle States Annual Conference December 10, 2010](https://reader036.vdocument.in/reader036/viewer/2022070305/5514b0f1550346b2598b62b6/html5/thumbnails/13.jpg)
Examples of Metrics Used for APR
![Page 14: The Academic Program Review Bridging Standards 7 and 14 Middle States Annual Conference December 10, 2010](https://reader036.vdocument.in/reader036/viewer/2022070305/5514b0f1550346b2598b62b6/html5/thumbnails/14.jpg)
TARGETS Caution Acceptable Aspirational
Graduation Rate
<19% 19%-23% >23%
![Page 15: The Academic Program Review Bridging Standards 7 and 14 Middle States Annual Conference December 10, 2010](https://reader036.vdocument.in/reader036/viewer/2022070305/5514b0f1550346b2598b62b6/html5/thumbnails/15.jpg)
![Page 16: The Academic Program Review Bridging Standards 7 and 14 Middle States Annual Conference December 10, 2010](https://reader036.vdocument.in/reader036/viewer/2022070305/5514b0f1550346b2598b62b6/html5/thumbnails/16.jpg)
TARGETS Caution Acceptable Aspirational
Transfer Rate <29% 29%-32% >32%
![Page 17: The Academic Program Review Bridging Standards 7 and 14 Middle States Annual Conference December 10, 2010](https://reader036.vdocument.in/reader036/viewer/2022070305/5514b0f1550346b2598b62b6/html5/thumbnails/17.jpg)
Definitions of Success & Retention
Success=Grades of (A,B,C & P)/(A, B, C, D, P, D, F, & W)
Retention=Grades of (W)/(A, B, C, D, P, D, F, & W)
![Page 18: The Academic Program Review Bridging Standards 7 and 14 Middle States Annual Conference December 10, 2010](https://reader036.vdocument.in/reader036/viewer/2022070305/5514b0f1550346b2598b62b6/html5/thumbnails/18.jpg)
![Page 19: The Academic Program Review Bridging Standards 7 and 14 Middle States Annual Conference December 10, 2010](https://reader036.vdocument.in/reader036/viewer/2022070305/5514b0f1550346b2598b62b6/html5/thumbnails/19.jpg)
![Page 20: The Academic Program Review Bridging Standards 7 and 14 Middle States Annual Conference December 10, 2010](https://reader036.vdocument.in/reader036/viewer/2022070305/5514b0f1550346b2598b62b6/html5/thumbnails/20.jpg)
Added a curricular analysis
• How well program goals support the college’s mission.
• How well individual course outcomes reinforce program outcomes.
• How well instruction aligns with the learning outcomes.
![Page 21: The Academic Program Review Bridging Standards 7 and 14 Middle States Annual Conference December 10, 2010](https://reader036.vdocument.in/reader036/viewer/2022070305/5514b0f1550346b2598b62b6/html5/thumbnails/21.jpg)
• Specific assessment results.
• Changes made based on the assessment findings.
• Evidence of closing the loop
• Changes made to the assessment plan.
![Page 22: The Academic Program Review Bridging Standards 7 and 14 Middle States Annual Conference December 10, 2010](https://reader036.vdocument.in/reader036/viewer/2022070305/5514b0f1550346b2598b62b6/html5/thumbnails/22.jpg)
Action Plan
• Outcomes expected as a result of appropriate actions steps.
• Timelines and persons responsible for each action step.
• Resources needed with specific budget requests.
• Evaluation plan with expected benefits.
![Page 23: The Academic Program Review Bridging Standards 7 and 14 Middle States Annual Conference December 10, 2010](https://reader036.vdocument.in/reader036/viewer/2022070305/5514b0f1550346b2598b62b6/html5/thumbnails/23.jpg)
Bottom Line
• Is there sufficient evidence that the program learning outcomes are being met?
• Is there sufficient evidence that the program is aligned with the college on specific key indicators?
![Page 24: The Academic Program Review Bridging Standards 7 and 14 Middle States Annual Conference December 10, 2010](https://reader036.vdocument.in/reader036/viewer/2022070305/5514b0f1550346b2598b62b6/html5/thumbnails/24.jpg)
The Framework
STANDARDS 1-7 Focus on Institutional Performance
Standard 7: Assessment of Institutional Effectiveness How well are we collectively doing what we say we are doing, with a specific focus on supporting student learning. Assessment must be included in Standards 1-6.
STANDARDS 8-14 Focus on Student Performance
Standard 14: Assessment of Student Learning Do we have clearly articulated learning goals, offer appropriate learning activities, assess student achievement of those learning outcomes, and use the result of assessment to improve teaching and learning and inform budgeting and planning. Also in Standards 8-13.
Strategic Analysis
Institutional data Link to the IEM
through use of common data sets
Curriculum Analysis Course assessment Program Assessment Core assessment
Planning and Budgeting (Standard 2)APR Action PlanAPR Annual ReportAnnual Academic Planning
Assessment Results
Curriculum Committee
President’s Office
Curriculum BOT & BOT
![Page 25: The Academic Program Review Bridging Standards 7 and 14 Middle States Annual Conference December 10, 2010](https://reader036.vdocument.in/reader036/viewer/2022070305/5514b0f1550346b2598b62b6/html5/thumbnails/25.jpg)
Addition of Technology
• Worked in concert with Information Technology to integrate iStrategy with ERP (Datatel).
• The implementation of this permitted end users to obtain the data needed for program assessment, without the middle man (IR and/or IT).
![Page 26: The Academic Program Review Bridging Standards 7 and 14 Middle States Annual Conference December 10, 2010](https://reader036.vdocument.in/reader036/viewer/2022070305/5514b0f1550346b2598b62b6/html5/thumbnails/26.jpg)
![Page 27: The Academic Program Review Bridging Standards 7 and 14 Middle States Annual Conference December 10, 2010](https://reader036.vdocument.in/reader036/viewer/2022070305/5514b0f1550346b2598b62b6/html5/thumbnails/27.jpg)
![Page 28: The Academic Program Review Bridging Standards 7 and 14 Middle States Annual Conference December 10, 2010](https://reader036.vdocument.in/reader036/viewer/2022070305/5514b0f1550346b2598b62b6/html5/thumbnails/28.jpg)
![Page 29: The Academic Program Review Bridging Standards 7 and 14 Middle States Annual Conference December 10, 2010](https://reader036.vdocument.in/reader036/viewer/2022070305/5514b0f1550346b2598b62b6/html5/thumbnails/29.jpg)
![Page 30: The Academic Program Review Bridging Standards 7 and 14 Middle States Annual Conference December 10, 2010](https://reader036.vdocument.in/reader036/viewer/2022070305/5514b0f1550346b2598b62b6/html5/thumbnails/30.jpg)
Next Steps in the Evolution of of College and Program Outcomes
![Page 31: The Academic Program Review Bridging Standards 7 and 14 Middle States Annual Conference December 10, 2010](https://reader036.vdocument.in/reader036/viewer/2022070305/5514b0f1550346b2598b62b6/html5/thumbnails/31.jpg)
Example of APR Report Card
Persistence Fall to Fall
2005 2006 2007 Full-time
Persistence Program 75% 50% 70% College 64.39% 63.16% 50%
Part-time Persistence
Program 25% 43.48% 33.33% College 33.52% 37.09% 34.41%
Persistence - These data are based on a cohort of first time students from a specific semester and follows their enrollment patterns one year out
Fall to Spring 2006 2007 2008
Full-time Persistence
Program 70% 80% 76.92% College 78.19% 80.98% 80.93%
Part-time Persistence
Program 69.57% 45.83% 46.15% College 45.12% 41.60% 39.00%
Persistence - These data are based on a cohort of first time students from a specific semester and follows their enrollment patterns from Fall semester to the following Spring semester
Graduation and Transfer
Degrees Conferred
2006 2007 2008 Three Year
Percent Change
Program 9 9 8 -11.11% College 1058 1046 1127 6.52%
Degrees Conferred – These are the actual number of degrees conferred, not the degrees earned
Pass Rates on Licensure Exams
Graduation and Transfer Rate: Cohort Entering 2005
Total
Students Degrees
Graduation Rate
(Within 3 Years) Transfer
Transfer Rate
(Within 3 Years)
Full-time Program 14 1 7.14% 5 35.71% College 1524 192 12.60%* 402 26.38%
Part-time Program 9 0 0% 4 44.44% College 990 26 2.62% 232 23.43%
First time students to College, no transfer credits *Acceptable 19-23%
Fiscal
Evidence of High Priority and Employment Predictions
![Page 32: The Academic Program Review Bridging Standards 7 and 14 Middle States Annual Conference December 10, 2010](https://reader036.vdocument.in/reader036/viewer/2022070305/5514b0f1550346b2598b62b6/html5/thumbnails/32.jpg)
• Examples of Course Success
![Page 33: The Academic Program Review Bridging Standards 7 and 14 Middle States Annual Conference December 10, 2010](https://reader036.vdocument.in/reader036/viewer/2022070305/5514b0f1550346b2598b62b6/html5/thumbnails/33.jpg)
Success in ACC 111
2003/FA 2004/FA 2005/FA 2006/FA
2007/FA
2008/FA
2009/FA
% Success 61.4% 57.1% 55.4% 55.3% 51.4% 44.2% 48.3%# Success 329 276 253 281 261 244 249% Non Success 38.6% 42.9% 44.6% 44.7% 48.6% 55.8% 51.7%# Non Success 207 207 204 227 247 308 267
![Page 34: The Academic Program Review Bridging Standards 7 and 14 Middle States Annual Conference December 10, 2010](https://reader036.vdocument.in/reader036/viewer/2022070305/5514b0f1550346b2598b62b6/html5/thumbnails/34.jpg)
Success in ACC 111
2003/FA 2004/FA 2005/FA 2006/FA 2007/FA 2008/FA 2009/FA
% Female Success 63.3% 57.5% 58.8% 57.7% 57.3% 51.8% 58.7%
Female Success 145 111 104 123 114 115 105
% Male Success 59.8% 56.8% 53.2% 53.6% 47.2% 39.1% 42.1%
Male Success 180 163 149 158 145 127 133
![Page 35: The Academic Program Review Bridging Standards 7 and 14 Middle States Annual Conference December 10, 2010](https://reader036.vdocument.in/reader036/viewer/2022070305/5514b0f1550346b2598b62b6/html5/thumbnails/35.jpg)
Success in Math 010
2003/FA 2004/FA 2005/FA 2006/FA 2007/FA 2008/FA 2009/FA% Success 53.6% 46.3% 47.3% 45.7% 44.8% 43.3% 47.4%
Success 310 266 276 293 297 288 344
% Non Success 46.4% 53.7% 52.7% 54.3% 55.2% 56.7% 52.6%
Non Success 268 309 307 348 366 377 381
![Page 36: The Academic Program Review Bridging Standards 7 and 14 Middle States Annual Conference December 10, 2010](https://reader036.vdocument.in/reader036/viewer/2022070305/5514b0f1550346b2598b62b6/html5/thumbnails/36.jpg)
Success in Math 010
2003/FA 2004/FA 2005/FA 2006/FA 2007/FA 2008/FA 2009/FA% African American Success
42.6% 37.7% 38.5% 25.8% 26.9% 29.9% 34.7%
African American Success
43 46 40 33 45 56 51
% Caucasian Success
58.2% 51.8% 50.3% 52.7% 53.5% 48.4% 52.0%
Caucasian Success 202 184 180 217 206 180 141
![Page 37: The Academic Program Review Bridging Standards 7 and 14 Middle States Annual Conference December 10, 2010](https://reader036.vdocument.in/reader036/viewer/2022070305/5514b0f1550346b2598b62b6/html5/thumbnails/37.jpg)
Benefits
• Build a bridge between Standards 7 and 14.
• Better data.
• By putting data in the hands of faculty, have them actively engaged with using data in decisions/planning.
• IR time better used.
![Page 38: The Academic Program Review Bridging Standards 7 and 14 Middle States Annual Conference December 10, 2010](https://reader036.vdocument.in/reader036/viewer/2022070305/5514b0f1550346b2598b62b6/html5/thumbnails/38.jpg)
• Annual planning cycle developed.• Built a culture of assessment in
several of the academic divisions.• Curricular changes that align with
graduation initiative.• Curricular and program
improvement.• Created a college-wide model for
improvement of student learning.
![Page 39: The Academic Program Review Bridging Standards 7 and 14 Middle States Annual Conference December 10, 2010](https://reader036.vdocument.in/reader036/viewer/2022070305/5514b0f1550346b2598b62b6/html5/thumbnails/39.jpg)
Evolution of the Dashboard
• Creation of a Student Success DashboardMetrics:
Course level success and retention (Developmental and College-Level)
Persistence (fall to spring and fall to fall) Progression of various cohorts of students College level success in Math or English
after Developmental Math or English Graduation Transfer
![Page 40: The Academic Program Review Bridging Standards 7 and 14 Middle States Annual Conference December 10, 2010](https://reader036.vdocument.in/reader036/viewer/2022070305/5514b0f1550346b2598b62b6/html5/thumbnails/40.jpg)
Graphic Representation for the SSD
![Page 41: The Academic Program Review Bridging Standards 7 and 14 Middle States Annual Conference December 10, 2010](https://reader036.vdocument.in/reader036/viewer/2022070305/5514b0f1550346b2598b62b6/html5/thumbnails/41.jpg)
Graphic Representation for the SSD
![Page 42: The Academic Program Review Bridging Standards 7 and 14 Middle States Annual Conference December 10, 2010](https://reader036.vdocument.in/reader036/viewer/2022070305/5514b0f1550346b2598b62b6/html5/thumbnails/42.jpg)
Final Thoughts
It’s not perfect, but it works for us. Do the research on which tools
are appropriate for your college Assessment of the core curriculum Launching of assessment software It all starts with asking the right
question PRR 2010
![Page 43: The Academic Program Review Bridging Standards 7 and 14 Middle States Annual Conference December 10, 2010](https://reader036.vdocument.in/reader036/viewer/2022070305/5514b0f1550346b2598b62b6/html5/thumbnails/43.jpg)
Questions
![Page 44: The Academic Program Review Bridging Standards 7 and 14 Middle States Annual Conference December 10, 2010](https://reader036.vdocument.in/reader036/viewer/2022070305/5514b0f1550346b2598b62b6/html5/thumbnails/44.jpg)
Presenters
Mr. H. Leon Hill
Director of Institutional Research
Dr. Joan E. Brookshire
Associate Vice President of Academic Affairs