Welcome

Cañada CollegeFlex Day

March 9, 2012

A Review of Best Practices

Based on a review of the literature and evidence from the site visit teams connected with four in-depth research investigations: Completion by Design, Achieving the Dream, the Aspen Prize research and the California BRIC Initiative.

The 50+ Community Colleges examined include both large and small colleges; those in urban, suburban & rural locations; those serving high percentages of both traditional and non-traditional students, those experiencing budget growth and budget decline.

Emphasis on PathwaysStrong peer support systemsOngoing professional

development

Frequent dialogue across silosHighly focused set of institutional priorities

Entrepreneurial in finding resources/funding

High degree of teacher-learner contact

A focus on equity & closing the achievement gapFrequent & in-depth dialogue

with employers

Evidence informed decision making at

every level

Strong linkages between planning & budgetingA focus on student completion

Smart investments in classroom technology

Strong ties to communityRoutine tracking of student

performance

Bringing successful pilot programs to scale

Effective basic skills education

Mandatory orientation & advising

Review of site visit documentation led to the creation of a set of categories that each college was scored against

Identifying the recurring

themes

The following categories represent the recurring themes across all the evidence reviewed.

1. Use of Evidencea) Providing access to quality, actionable datab) Ensuring multiple groups review & act on datac) Use Data to guide planning, budgeting & Program designd) Collect evidence for cost-benefit analysis

2. Coherent, Structured & Systematic Approacha) Understanding student pathwaysb) Focusing classroom, support, structures & prof dev on student outcomesc) Providing sustainable, tailored support to studentsd) Integration of efforts across divisions

3. Equity in Student Access & Achievementa) Access for low-income & underreresented students to quality programsb) Leadership’s advocacy for equitable outcomesc) Review of student data by race & ethnicity for action planningd) Using experiences of underrepresented students for planning

4. Scalability & Sustainability of Approachesa) Professional development & succession planning for sustainabilityb) Institutionalizing successful approaches without relying on soft moneyc) Systematic consideration of whether all populations are being supportedd) Using cost benefit analysis to asses programs

5. Leadershipa) Personal investment of institutional leaders in student successb) Cultivating shared ownership of student success across the institutionc) Rewarding actionable inquiry & experimentationd) Intentional prioritization of concrete, sustainable initiatives

But a review of the characteristics displayed by the highest performing colleges reveals a short list of capabilities associated with sustained success

1. Rewarding actionable inquiry & experimentation

2. Intentional prioritization of initiatives

3. Ensuring multiple groups review & act on data 4. Clear pathways & coherent structures

5. Evidence-rich conversations with employers and the community

A few comments on the use of

evidence

1,000 ftPerspective

100 ftPerspective

On the Ground

Institutional Strategies

Classroom Innovation

Program Improvements

Review of Best Practices is most meaningful when it informs decision making at the proper place of practice

Best Practices to improve the graduation rate for degree or certificate seeking students Best Practices for

helping students completing course sequences within in programs

Best Practices to support the development of new course methodology to improve student success

Desired Outcome

Starting Point

Experimentation does not always yield shortest-path solutions

Acknowledge the limits of data and the importance of experience

Data rarely enter a vacuum and suddenly illuminate a hidden truth that triggers action.

Data join the narrative that experts have developed through years of observation & practice.

The narrative explains and reacts to the data and the two coevolve to shed new light on the issues we study.

The process of inquiry is not a search for an absolute truth

Domain of possible solutions

We answer the questions that eliminate dead end solutions

And what do we do when the evidence is ambiguous?

Gregory M Stoup, Cañada College

What to do when you reach the limits of your research and yet still face multiple choices?

Trust your experience, your

intuition & choose !

Program Learning Outcomes

Programs, Student Goals & Learning Outcomes

Student Goals

• Transfer to 4-year

• Earn AA/AS Degree

• Earn Certificate

• Take various courses for a promotion

• Take a single course to upgrade a job skill

• Personal enrichment

• Others?

• Transfer to 4-year

• Earn AA/AS Degree

• Earn Certificate

• Take various courses for a promotion

• Take a single course to upgrade a job skill

• University student course takers

• Personal enrichment

• Others?

The idea: Institutional & Program Level Assessment helps determine whether students can integrate learning from individual courses into a coherent whole*

Program Learning Outcomes

Addressed through Course Level Assessment

GE Learning Outcomes

* Source: Palomba, C.A. & Banta T.W. 2009. Assessment Essentials. San Francisco: Jossey-Bass

Cañada College Programs

• Engineering/CIS• Mathematics• Physical Sciences (physics,

chemistry, astronomy, earth science)

• Biological Sciences (biology, health sciences)

• Social Sciences (Anthropology, Political Science, History, Economics, Philosophy, Psychology, Sociology, Geography)

• Language Arts (English, Literature)• Fine & Performing Arts (Art, Art

History, Music, Theater)• ESL• Basic Skills

• Foreign Language• Communication Studies• Design, Technology &

Creative Arts (Interior Design, Fashion, MART, Architecture)

• Kinesiology & Athletics (PE, Fitness, Athletics, Dance)

• Business & Accounting • Medical Assisting• CBOT• Early Childhood Education /

Child Development• Paralegal• Human Services• Honors Transfer

Alternatively, faculty can assemble samples of student work from various classes and use the collective work sample to assess specific program learning outcomes.

Program Portfolios – Once students enter a program they are instructed to retain course work from multiple courses in that program. The collective body of work should related to the PLOs. That body of work is then reviewed by faculty and assessed against specific PLOs.

Options to Consider

How should we assess our Program Learning Outcomes?

Student Presentations: Observations of any behavior such as performance, project presentations, oral exams, recitals, etc. are evaluating using a locally develop rubric.

How should we assess our Program Learning Outcomes?

Scoring Rubrics: Rubrics can be used to score any product or performance. The rubric delineates the criteria used to discriminate among levels of achievement. Generally two raters are used to review each student product and a third rater is used to resolve discrepancies.

How should we assess our Program Learning Outcomes?

Embedded Questions on Assignments or Exams: Questions related to program learning outcomes can be embedded within course assignments or exams.

For example, a final exam in a higher level course in a program could include a question or set of questions relating to your program SLOs. Faculty grade the exams as usual and then separate exam questions that are linked to the program SLOs for analysis. The findings are reported as an aggregate.

Locally Developed Final Exams: Faculty create an objective exam for graduating students that is aligned with the program SLOs.

How should we assess our Program Learning Outcomes?

How should we assess our Program Learning Outcomes?

Other options:

Locally Developed Final Exams

Pre-Test/Post Test Evaluations

Student Exit Interviews/Survey/Focus Groups*

* While this is an indirect method of assessment, this might be a good option for programs having a small number of graduates in a given assessment cycle.

Program Learning Outcomes have been created

Develop or identify a tool for assessing them

Collect some evidence

Review the evidence

They are published & the students are aware of them

This could be any one or more of the PLOs on our list

You have looked at how students performed on their PLOs (possibly against a rubric) and documented that information

You have collected something in the way of output/evidence from students that are tied to your PLOs

Stage in the process What this means to usAfter looking at the evidence you make a determination that:

a) The PLO didn’t really get to the heart of what you had in mind and you decide to modify your PLO

b) The results were a little “iffy” and you would like to collect more evidence before drawing conclusions

c) Your expectations were met or exceeded. In discussion with colleagues you decide to continue to examine the same PLO next year or possibly develop another PLO

d) Your expectations weren’t met and you have conversations with your colleagues on how to address it.

Respond to the evidence with some action

Course Alignment within Programs

Also subject to review by accrediting teams

Generated by our TrakDat software

A supplement to direct methods of Program Learning Outcomes assessment (by itself not sufficient as evidence of PLO performance)

Explain how the elements of form and principles of design work together with the creative process to produce a work of art.

Define major historical and contemporary movements in art and discuss how art reflects its time and culture.

Identify how an appreciation of the visual arts’ influences the quality of life.

Communication Skills

Critical & Creative Thinking

Understanding Society & Culture

Scientific & Quantitative

Reasoning

Visual Analysis: Ability to recognize, distinguish, and characterize styles of objects, artists, movements, media

World Cultures: Knowledge about cultures remote from students’ own, attained through study of works of art produced by those cultures.

Writing Skills: Skills for writing critically and persuasively, with specific application to art history.

Program Learning Outcomes: Art history

Institutional Learning Outcomes: General Educational

Course Learning Outcomes: Art 101 (Intro to Visual Arts)

Visual Analysis: Ability to recognize, distinguish, and characterize styles of objects, artists, movements, media

World Cultures: Knowledge about cultures remote from students’ own, attained through study of works of art produced by those cultures.

Writing Skills: Skills for writing critically and persuasively, with specific application to art history.

Identify how an appreciation of the visual arts’ influences the quality of life.

Define major historical and contemporary movements in art and discuss how art reflects its time and culture.

Explain how the elements of form and principles of design work together with the creative process to produce a work of art.

Wrapping up

Any questions?