setting institutional standards for student achievement: meeting the call for academic quality
Post on 04-Jun-2018
217 Views
Preview:
TRANSCRIPT
-
8/13/2019 Setting Institutional Standards for Student Achievement: Meeting the Call for Academic Quality
1/49
Setting Institutional Standards for Student
Achievement:
Meeting the Call for Academic Quality
Fred Trapp, Cambridge West Partners
Carolyn Arnold, Chabot College
Daylene Meuschke, College of the Canyons
Elaine Kuo andAndrew LaManque, Foothill DeAnza Community
College District
Robert Pacheco, MiraCosta College
-
8/13/2019 Setting Institutional Standards for Student Achievement: Meeting the Call for Academic Quality
2/49
Some RP Resources
Learning Assessment Listserv (www.cccnext.net)
RP Group Listserv (www.cccnext.net)
Perspectives (www.rpgroup.org)
Cloud Housing Session
Docs http://tinyurl.com/mjdu9ha
http://www.cccnext.net/http://www.cccnext.net/http://www.rpgroup.org/http://tinyurl.com/mjdu9hahttp://tinyurl.com/mjdu9hahttp://tinyurl.com/mjdu9hahttp://www.rpgroup.org/http://www.cccnext.net/http://www.cccnext.net/ -
8/13/2019 Setting Institutional Standards for Student Achievement: Meeting the Call for Academic Quality
3/49
Outcomes for the Session
Discover some best practices in setting college-
wide standards for achievement.
Brainstorm ways to integrate the standards intoplanning and decision-making processes at your
institution.
Evaluate the reasonableness of your colleges
institution-set standards.
-
8/13/2019 Setting Institutional Standards for Student Achievement: Meeting the Call for Academic Quality
4/49
How Well Roll
Trends and a Literature Review (10-15 minutes each)
Three College/District Stories (10-15 minutes each)
Interactive Portion (25 minutes)
Reflect on your own institution, its culture, history, mission to consider yourown institution-set standards. Compare, borrow, validate
-
8/13/2019 Setting Institutional Standards for Student Achievement: Meeting the Call for Academic Quality
5/49
Institution SetStandards: Trends &
Literature
Frederick Trapp
Cambridge West Partnership, LLC
Long Beach City College (retired)
October 2013
-
8/13/2019 Setting Institutional Standards for Student Achievement: Meeting the Call for Academic Quality
6/49
Professional Literature/National Trends
The three Es of public agencies
1. Economy
2. Efficiency
3. Effectiveness
How high or low is the effectiveness
bar is set to define quality service?
-
8/13/2019 Setting Institutional Standards for Student Achievement: Meeting the Call for Academic Quality
7/49
ACCJC Annual Report, 2013
Student achievement performance metrics
14b Successful student course completion rate
15b Student retention percentage
16b Student degree completion
17b Student transfer to 4-year colleges
18b Student certificate completion
A standard is the level of performance set by the institution
to meet educational quality and institutional effectiveness
expectations. This number may differ from a performance
improvement goal which an institution may aspire to meet.
-
8/13/2019 Setting Institutional Standards for Student Achievement: Meeting the Call for Academic Quality
8/49
Standard I.B.2
The institution sets goals to improve its effectiveness consistent
with its stated purposes. The institution articulates its goals and
states the objectives derived from them in measurable terms so
that the degree to which they are achieved can be determined
and widely discussed.
-
8/13/2019 Setting Institutional Standards for Student Achievement: Meeting the Call for Academic Quality
9/49
Standard I.B.3
The institution assesses progress toward achieving its stated
goalsand makes decisions regarding the improvement of
institutional effectiveness in an ongoing and systematic cycle of
evaluation, integrated planning, resource allocation,
implementation, and re-evaluation.
How bad does performance have to get before we
decide it is unacceptable and we need to do something?
-
8/13/2019 Setting Institutional Standards for Student Achievement: Meeting the Call for Academic Quality
10/49
US Dept. of Ed., New Regulations
34 Code of Federal Regulations (CFR) & USDE Guidelines for
602.16(a)(1)(i).
Standards effectively address success with respect to student
achievement in relation to the institutions mission, which may
include different standards for different institutions or programs,as established by the institution, including, as appropriate,
consideration of course completion, State licensing examinations,
and job placement rates.
Whether institutionally-developed standards to demonstratestudent success are being used by the accreditorin the
accreditation assessment, and the institutions performance with
respect to student achievement is assessed.
-
8/13/2019 Setting Institutional Standards for Student Achievement: Meeting the Call for Academic Quality
11/49
ACCJC Direction to Teams
Evaluation teams will examine student achievement data at
the programmatic and institutional levels. The institution must
set standards of satisfactory performance for student
achievement and evaluate itself against those standards at
both levels.1. Examine the institutional set-standards for student
achievement and assess their appropriateness
(reasonableness)
2. Consider the set-standards in the context of satisfactory
performance, goals for improvement (achievement and
learning).
3. Cite this information as evidence in describing its evaluation
of how well the institution accomplishing its mission.
-
8/13/2019 Setting Institutional Standards for Student Achievement: Meeting the Call for Academic Quality
12/49
ACCJC Direction to Teams
Describe in the team report
Not the basis of a recommendation or
a sanction
-
8/13/2019 Setting Institutional Standards for Student Achievement: Meeting the Call for Academic Quality
13/49
Health Care- Setting Standards
National Quality Forum (1999)
700 best in class quality measures, rooted in evidence-based
practice
Medical Loss Ratio (80-85% of premium goes to care)
Centers for Medicare & Medicaid Services (CMS)- AccountableCare Organizations (ACOs) initiative performance domains
Patient/caregiver experience
Care coordination/patient safety
Preventive care At-risk populations
Moving from fee-for-service to pay-for-performance using
national performance benchmark data and a sliding scale.
-
8/13/2019 Setting Institutional Standards for Student Achievement: Meeting the Call for Academic Quality
14/49
Performance Scorecards and Dashboards
Strategicmeans visually to monitor achievement of objectives Rests on some cause and effect beliefs
Association for Institutional Research (AIR) study of 66 higher
education dashboards
Category % Using
Financial Indicators 80%
Admissions 79%
Enrollment 77%
Faculty 77%
Student Outcomes 72%
Student Engagement 58%Academic information 55%
Physical Plant 38%
Satisfaction 35%
Research 35%
External Ratings 21%
-
8/13/2019 Setting Institutional Standards for Student Achievement: Meeting the Call for Academic Quality
15/49
Setting the Performance Standard
Levels set by1. Benchmarking (www.NCCBP.org)
2. Industry standards
3. Recent performance history
5-year average
3-year rolling average
Color codes for performance (stop light)
Red- poor; yellow- below target; green- on target
Performance expressed as a range
=>5% above- very good
=
-
8/13/2019 Setting Institutional Standards for Student Achievement: Meeting the Call for Academic Quality
16/49
Performance Funding for Higher Education
In place
30%
In
transitio
n8%
Considering
32%
No Action
30%
50 States
National Council of State Legislatures
-
8/13/2019 Setting Institutional Standards for Student Achievement: Meeting the Call for Academic Quality
17/49
Measuring Institutional Effectiveness
through Institutional Standards One
Colleges Approach
Daylene M. Meuschke, Ed.D.
Director, Institutional Research
College of the Canyons
Strengthening Student Success Conference
October 2013
-
8/13/2019 Setting Institutional Standards for Student Achievement: Meeting the Call for Academic Quality
18/49
The Plan / TimelineMarch 2013*Required to establishstandards
*Convened College PlanningTeam sub-group
*Established indicators andstandards
April-June 2013
*Presented Indicators-Senate,College Planning Team, Boardof Trustees
*Obtained feedback onPerformance Indicators
April 2013-present -*Held monthly meetings
*Using RP Groups Principles ofRedesign to guide planningactivities
Source: www.photo-dictionary.com
-
8/13/2019 Setting Institutional Standards for Student Achievement: Meeting the Call for Academic Quality
19/49
Source: www.2.acs.ncsu.edu
StudentLearning /
Institutional
Effectiveness
ACCJCAnnual
Survey
Scorecard
InstitutionalData
Existing Data Sources
http://www.2.acs.ncsu.edu/http://www.2.acs.ncsu.edu/ -
8/13/2019 Setting Institutional Standards for Student Achievement: Meeting the Call for Academic Quality
20/49
COCs Performance Indicators
and Institutional Standards
Coming soonMetric for Career Technical Education
-
8/13/2019 Setting Institutional Standards for Student Achievement: Meeting the Call for Academic Quality
21/49
Planning Guided by RP Groups
Principles of Redesign1. Accelerate entry into coherent programs of study
2. Minimize time required to get college-ready
3. Ensure students know the requirements to
succeed4. Customize and contextualize instruction
5. Integrate student support with instruction
6. Continually monitor student progress and
proactively provide feedback
7. Reward behaviors that contribute to completion
8. Leverage technology to improve learning and
program delivery
-
8/13/2019 Setting Institutional Standards for Student Achievement: Meeting the Call for Academic Quality
22/49
Infusing the Standards
College-wide
College Planning Team
Performance Indicators Sub-Committee
Skills for Success CommitteeRoadmap Project (AAC&U)
Grants (e.g. Title III)
Departments
Academic and Administrative Program Reviews
Documented in Research Brief #57
More to Come!
-
8/13/2019 Setting Institutional Standards for Student Achievement: Meeting the Call for Academic Quality
23/49
Next Steps:
Revise Indicators
Add CTE
Revise Standards Completion (success)
Continue Disaggregating
Connect to departments
Some are clean, clear Others (e.g., transfer)-
more problematic
Source: www.burkemuseum.org
Just scratching the surface
Plan
ImplementEvaluate
http://www.burkemuseum.org/static/polarbear/biblio.phphttp://www.burkemuseum.org/static/polarbear/biblio.php -
8/13/2019 Setting Institutional Standards for Student Achievement: Meeting the Call for Academic Quality
24/49
Any questions?
Thank you!
-
8/13/2019 Setting Institutional Standards for Student Achievement: Meeting the Call for Academic Quality
25/49
Measuring up to the StandardsCreating Meaningful Student Groups to measure
progress towards Institutional Standards
Carolyn Arnold, Ph.D.
Coordinator, Office of Institutional Research
Planning, Review, and Budget Council (PRBC)
Chabot College
-
8/13/2019 Setting Institutional Standards for Student Achievement: Meeting the Call for Academic Quality
26/49
Chabot Institutional Standards
Setting the standards
Analyzed long term trends for the 5 standards
Chose five year rolling averages Reflect recent trends
Smooth out extreme highs or lows
Sometimes above standard; sometimes below
Assumption no matter where we set them How can we improve these outcomes?
-
8/13/2019 Setting Institutional Standards for Student Achievement: Meeting the Call for Academic Quality
27/49
Chabot Institutional Standards
Challenge 1 to improving the outcomes
Degrees, Transfers, and Certificates are long-term outcomes Take semesters or years to complete Completion numbers fluctuate year to year
Most college initiatives focus on short termobjectives: More students taking basic skills courses
Increasing success in College English, Math
How can short and long term be connected?
-
8/13/2019 Setting Institutional Standards for Student Achievement: Meeting the Call for Academic Quality
28/49
Chabot Institutional Standards
Challenge 2 to improving the outcomes
Not all students aiming for these outcomes Have different goals
Going towards goals at different speeds
Starting from different places
How can we take into account student differencesin goals, speed, and starting places when we
attempt to improve outcomes? We do want to improve!
-
8/13/2019 Setting Institutional Standards for Student Achievement: Meeting the Call for Academic Quality
29/49
Chabot Strategic Plan Goal
Increase the number of students who achieve theireducational goal in a reasonable time (outcomes) Institutional Research supposed to measure the number
by clarifying pathways and providing moreinformation and support (methods) College initiatives supposed to improve the number
-
8/13/2019 Setting Institutional Standards for Student Achievement: Meeting the Call for Academic Quality
30/49
Not One Number!
Why we created distinct student groups
To take into account student differences in goals, speed, levels
To monitor progress and outcomes of studentcohorts with different educational goals
To determine reasonable time to completion foreach group
To target college initiatives and support servicesneeded by each group
-
8/13/2019 Setting Institutional Standards for Student Achievement: Meeting the Call for Academic Quality
31/49
Creating educational goal
groups for new students
Groups based on:
Their educational goal Self-reported educational goal
How long they will take
How many units they take (FT/PT) Where they start (assessment in English)
-
8/13/2019 Setting Institutional Standards for Student Achievement: Meeting the Call for Academic Quality
32/49
Creating
educational goal groups
Laser(FT) College
Laser(FT) Basic Skills
Laser(FT) Not Assessed
Seeker (PT) College
Seeker(PT) Basic Skills
Seeker (PT) Not Assessed
Explorer
Career-builder FT
Career-builder PT
Skills-builder
Student Ed Goal GroupsEngl ish
AssessmentUnitsEd Goal
College
Basic SkillsNot Assessed
College
Basic Skills
Not Assessed
Full-
time
Part-
time
6+ units
Transfer
or
Degree
(GE)
UndecidedCertificate or
Job training
Cert/Job/Und/
Pers Devel
Full-time
Part-time 6-11 units
Under 6 units
-
8/13/2019 Setting Institutional Standards for Student Achievement: Meeting the Call for Academic Quality
33/49
10 student
educational goal groups Laser (FT) College
Laser (FT) Basic Skills
Laser (FT) Not Assessed
Seeker (PT) College
Seeker (PT) Basic Skills
Seeker (PT) Not Assessed
Explorer
Career-builder FT
Career-builder PT
Skills-builder
-
8/13/2019 Setting Institutional Standards for Student Achievement: Meeting the Call for Academic Quality
34/49
Chabot Fall 2012 New StudentsPercentage in each Educational Goal Group
8%
22%
3%
5%
25%
10%
10%
2%
3%
12%
Laser FT College
Laser FT BasSkls
Laser FT NoAssess
Seeker PT College
Seeker PT BasSkls
Seeker PT NoAssess
Explorer
Career Builder FT
Career Builder PT
Skills Builder
-
8/13/2019 Setting Institutional Standards for Student Achievement: Meeting the Call for Academic Quality
35/49
Student ed goal groups:
Look and behave differently by
Age Gender
Race-ethnicity Day vs. evening
Early Decision
Financial Aid
Pct assessed in Math Math assessment levels
Types of major
-
8/13/2019 Setting Institutional Standards for Student Achievement: Meeting the Call for Academic Quality
36/49
Student ed goal groups:
Taking differences into account How do we know what is the reasonable
time it will take each group to achieve their
educational goal? First, answer challenge #1 to meeting the
standards outcomes
Connect long term outcomes with short termobjectives by setting milestones
-
8/13/2019 Setting Institutional Standards for Student Achievement: Meeting the Call for Academic Quality
37/49
Student ed goal groups
progress at different rates Early Engagement
Enrolled in basic skills
Momentum
Completed basic skills Persistence
Progress Completed College Math or English
Earned Transferable units
Earned CTE units
Completion Earned certificates or degrees
Transfer ready / transferred
Employed
-
8/13/2019 Setting Institutional Standards for Student Achievement: Meeting the Call for Academic Quality
38/49
Student ed goal groups
allow us to: Get beyond one number for each
outcome.
10 student groups Measure progress along the way.
Progress milestones
Determine reasonable time to milestones
and completion for each group based onpast cohorts
Tailor college initiatives for each group andsee effect of initiatives.
-
8/13/2019 Setting Institutional Standards for Student Achievement: Meeting the Call for Academic Quality
39/49
Strengthening Student Success Conference 2013
Elaine Kuo, Ph.D.Andrew LaManque, Ph.D.
Setting ACCJC Institutional
Standards at Foothill College
-
8/13/2019 Setting Institutional Standards for Student Achievement: Meeting the Call for Academic Quality
40/49
Legal Foundation Summary
The requirement for Institutional Standards flow from
Federal Regulations
The Regulations are not new (2009) but the
Commission has stated that the Federal interpretationchanged last year.
The new interpretation was included in evaluation team
training in Spring 2013.
40
-
8/13/2019 Setting Institutional Standards for Student Achievement: Meeting the Call for Academic Quality
41/49
The Accreditation Standards Process
Colleges setInstitutionalStandards
ACCJC collectsannual data onInstitutionalStandards
Visiting teamsevaluate whetherStandards are met
and reasonable
Recommendationsto meet the
standards or makethem reasonable?
41
-
8/13/2019 Setting Institutional Standards for Student Achievement: Meeting the Call for Academic Quality
42/49
The Process
Accreditation team
Shared governance groups
Planning and ResourceCouncil (PaRC)
Academic Senate
Classified Senate
Imbed in college planning,
program review
The Data
Most recent term or year
Fall 2012 or 2011-12
Longitudinal2007-08 to 2011-12
Disaggregation
Ethnicity
ProgramSources
CCCCO data mart
FHDA IR&P
42
Foothill Approach
-
8/13/2019 Setting Institutional Standards for Student Achievement: Meeting the Call for Academic Quality
43/49
The Data: Setting the Standards
Completion based on passingcourse with A, B, C, or P grade;includes all courses offered in Fallterm
Table 1. Student Course Completion Rate
Fall 2012 75%
Five-Year Average 76%
Five-Year Average by Ethnicity-Low 61%
Five-Year Average by Program-Low 60%
Institutional Standard Recommendation 55%
Source: CCCCO Data Mart, FHDA IR&P
Fall rates: 2011: 74%
2010: 76%
2009: 77%
2008: 79%43
-
8/13/2019 Setting Institutional Standards for Student Achievement: Meeting the Call for Academic Quality
44/49
The Process: Shared Governance Review
DiscussmethodologywithAccreditationTeam
Presentation anddiscussion atmain shared
governance body
Methodology andrecommendationsaccepted by mainshared governancebody and collegepresident
44
Institutional-set standards
scheduled for annual review
Key players include:
ALO, VPI, IR,
Academic Senate
President, Classified
Senate President
-
8/13/2019 Setting Institutional Standards for Student Achievement: Meeting the Call for Academic Quality
45/49
The Process: Program Review
Shared Governance
Integrated Planning& Budget Taskforce
Program ReviewTemplates
Student equity andinstitutional
standards section
Program ReviewCommittee
45
Prompts include:
Comment on your programs course success data, including any differences in completionrates by student demographics as well as efforts to address these differences.
Data sources: Program review data sheets, CCCCO data mart
-
8/13/2019 Setting Institutional Standards for Student Achievement: Meeting the Call for Academic Quality
46/49
Institution and Program Standards
For discussion:
How should a department course success /
completion rate reported during program reviewrelate to the institution standard?
If the department is below the institutionalstandard should there be a plan for getting it up
to the institutional standard?
46
-
8/13/2019 Setting Institutional Standards for Student Achievement: Meeting the Call for Academic Quality
47/49
Shaping the Dialogue
Three Roles, One Product
Three Roles: Scrivener
Time Keeper
Facilitator
Product: Summary Sheet
Notes
Three takeaways from the discussion
One Muddiest Point
-
8/13/2019 Setting Institutional Standards for Student Achievement: Meeting the Call for Academic Quality
48/49
Shaping the Dialogue
How were the institutional set standards developed at
your institution?
How might these standards be integrated with your
colleges decision-making and planning processes?
How would you justify the reasonablenessof the
institutional set standards in the context of your mission?
-
8/13/2019 Setting Institutional Standards for Student Achievement: Meeting the Call for Academic Quality
49/49
Wrap Up:What Have We Learned?
Discover some best practices in setting college-wide
standards for achievement.
Brainstorm ways to integrate the standards into planning
and decision-making processes at your institution.
Evaluate the reasonableness of your colleges institution-
set standards.
top related