The The AcademicAcademic and and Student ServicesStudent Services Assessment ProcessAssessment Process

atat

CALIFORNIA BAPTIST UNIVERSITYCALIFORNIA BAPTIST UNIVERSITY

Note: This is an “overview” and many implementation details are not includedNote: This is an “overview” and many implementation details are not included

What is assessment and why should I even care?What is assessment and why should I even care?

Neal F. McBride, Ed.D., Ph.D.Professor and Associate Provost

Phil Martinez, M.A.Director of Assessment

Office of Institutional Research, Planning, and Assessment

?What is assessment within What is assessment within higher educationhigher education

(and here at CBU)(and here at CBU)

33

What is Assessment?What is Assessment?

Put simply, Put simply, assessmentassessment is collecting is collecting evidence (data) to ascertain to what degree evidence (data) to ascertain to what degree an academic or student life department is an academic or student life department is

meeting its stated student learning and meeting its stated student learning and institutional capacity (administrative) institutional capacity (administrative)

outcomes. The collected evidence provides outcomes. The collected evidence provides the basis for deciding on what the basis for deciding on what

improvements, if any, are warranted.improvements, if any, are warranted.

44

WHY DO ASSESSMENT?WHY DO ASSESSMENT?

Assessment is an important tool in helping us to:Assessment is an important tool in helping us to:

1. BETTER SERVE OUR STUDENTS BY PURSUING ACADEMIC EXCELLENCE

1. BETTER SERVE OUR STUDENTS BY PURSUING ACADEMIC EXCELLENCE

2. OBTAIN RELIABLE INFORMATION FOR MAKING DECISIONS (Budget, Personnel, Curriculum, etc.)

2. OBTAIN RELIABLE INFORMATION FOR MAKING DECISIONS (Budget, Personnel, Curriculum, etc.)

3. MEET CONSITUENCY AND ACCREDITATION EXPECTATIONS

3. MEET CONSITUENCY AND ACCREDITATION EXPECTATIONS

Why I Should CareWhy I Should Care

55

EDUCATIONAL EFFECTIVENESSEDUCATIONAL EFFECTIVENESS

HOW DO WE KNOW WE ARE DOINGWHAT WE SAY WE ARE DOING?

CLAIM: “We offer quality education.”How do we know this claim is true?

HOW DO WE KNOW WE ARE DOINGWHAT WE SAY WE ARE DOING?

CLAIM: “We offer quality education.”How do we know this claim is true?

Educational effectiveness is the common goal for all Educational effectiveness is the common goal for all CBU colleges, schools, departments, and programs CBU colleges, schools, departments, and programs Educational effectiveness is the common goal for all Educational effectiveness is the common goal for all CBU colleges, schools, departments, and programs CBU colleges, schools, departments, and programs

DemonstratingDemonstrating educational effectiveness is the educational effectiveness is the goal for assessment . . . collecting goal for assessment . . . collecting EVIDENCEEVIDENCEDemonstratingDemonstrating educational effectiveness is the educational effectiveness is the goal for assessment . . . collecting goal for assessment . . . collecting EVIDENCEEVIDENCE

THE BASIC, KEY QUESTION:THE BASIC, KEY QUESTION:

66

ASSESSMENT IS INTEGRAL TO HIGHER EDUCATIONASSESSMENT IS INTEGRAL TO HIGHER EDUCATIONFACT:FACT:

We have always done course-level student assessment, now we apply assessment

processes to academic and student services programs in order to pursue educational

effectiveness throughout CBU

We have always done course-level student assessment, now we apply assessment

processes to academic and student services programs in order to pursue educational

effectiveness throughout CBU

Assessment is NOT optional... It is a professional obligation

Assessment is not a passing fad... Holding your breath until it goes away won’t work

77

Assessment or Evaluation?Assessment or Evaluation?

Evaluation – The next logical step after assessment, evaluation seeks to make judgments and decisions, based on the assessment data, about the appropriate course of action (i.e., continue on as is, make indicated adjustments, add a needed element, and/or terminate all or part of the program/service) Functionally, evaluation is done as an integral step within assessment conducting by individual programs and/or departments, as well as by their respective schools, colleges, or student services leadership

Evaluation – The next logical step after assessment, evaluation seeks to make judgments and decisions, based on the assessment data, about the appropriate course of action (i.e., continue on as is, make indicated adjustments, add a needed element, and/or terminate all or part of the program/service) Functionally, evaluation is done as an integral step within assessment conducting by individual programs and/or departments, as well as by their respective schools, colleges, or student services leadership

88

A Side Comment About TermsA Side Comment About Terms

GOAL

OBJECTIVE

AIM

GOAL

OBJECTIVE

AIM

ALL EQUALALL EQUAL “OUTCOME”“OUTCOME”

A statement expressing intent

A statement expressing intent

We use “outcome” as a generic replacement for the other termsWe use “outcome” as a generic replacement for the other terms

99

Who is Responsible for Assessment?Who is Responsible for Assessment?

►► The appropriate VP and/or School/College DeanThe appropriate VP and/or School/College Dean

►► Assessment Coordinators in each school, college, Assessment Coordinators in each school, college, or student servicesor student services

►► Department Chair or Program DirectorDepartment Chair or Program Director

►► AllAll faculty and student services staff faculty and student services staff

Assessment is a Assessment is a sharedshared responsibilityresponsibility

No one person can do it alone!No one person can do it alone!

Successful assessment demands a team effortSuccessful assessment demands a team effort

1010

Doing assessment to collect data (evidence) on Doing assessment to collect data (evidence) on educational effectiveness is educational effectiveness is notnot a one-time a one-time

event or an isolated process, but is an on-going, event or an isolated process, but is an on-going, continuous expectationcontinuous expectation

Please Keep in Mind:Please Keep in Mind:

Assessment is a Assessment is a yearlyyearly activity that provides data activity that provides data for Program Review, which is a for Program Review, which is a periodicperiodic event event

1111

The Assessment Process at CBUThe Assessment Process at CBU

Step 1 - MISSION Mission Statement

Step 2 - OUTCOMES Student Learning Capacity Outcomes (Optional)

Step 3 - OVERALL PLAN Overall Plan

201X-201X Assessment Cycle Annual Plan Yearly Report

THE BASIC SYSTEM OR “SKELETON”THE BASIC SYSTEM OR “SKELETON”

1212

FIRST, LET’S LOOK ATTHE OVERALL

ASSESSMENT PROCESSDEPICTED IN A FLOW DIAGRAM

FIRST, LET’S LOOK ATTHE OVERALL

ASSESSMENT PROCESSDEPICTED IN A FLOW DIAGRAM

1313

The Basic Assessment ProcessThe Basic Assessment Process

CBU Missionand Priorities

andUniversity Student

Outcomes(Core 4)

CBU Missionand Priorities

andUniversity Student

Outcomes(Core 4)

Student LearningOutcomes

(including faith Integration)

and OptionalInstitutional Capacity

Outcomes

“Mapping”

Student LearningOutcomes

(including faith Integration)

and OptionalInstitutional Capacity

Outcomes

“Mapping”

WRITEWRITE

Given theGiven theCollege, School,

Department and/or

programMission

College, School,

Department and/or

programMission

WRITEWRITE

Continued. . .Continued. . .

1414

The Basic Assessment ProcessThe Basic Assessment Process

Student LearningOutcomes

(including faithintegration)

Optional Institutional Capacity

Outcomes

Maps

Student LearningOutcomes

(including faithintegration)

Optional Institutional Capacity

Outcomes

Maps

WRITEWRITE

Given theGiven the

“OPlan” Overall

AssessmentPlan

“OPlan” Overall

AssessmentPlan

WRITEWRITE

CONTINUEDCONTINUED

“APlan”Annual

AssessmentPlan

“APlan”Annual

AssessmentPlan

Continued. . .Continued. . .

1515

The Basic Assessment ProcessThe Basic Assessment Process

THEN WRITETHEN WRITE

CONTINUEDCONTINUED

YearlyAssessment

Report

YearlyAssessment

Report

DOASSESSMENT

Implement the Annual Assessment Plan; collect data (evidence); analyze

the data; makedecisions (evaluate); and

make any neededchanges/adjustments, etc.

DOASSESSMENT

Implement the Annual Assessment Plan; collect data (evidence); analyze

the data; makedecisions (evaluate); and

make any neededchanges/adjustments, etc.

Continued. . .Continued. . .

1616

The Basic Assessment ProcessThe Basic Assessment ProcessCONTINUEDCONTINUED

WRITEWRITE

As NeededAs Needed

CBU Program Reviewsor

WASC Self Studiesor

ProfessionalAccreditation Reports

orGrant Requests

Etc.

CBU Program Reviewsor

WASC Self Studiesor

ProfessionalAccreditation Reports

orGrant Requests

Etc.

YearlyAssessment

Reports

YearlyAssessment

Reports

Given theGiven the

What do you do with the evidence (data)?What do you do with the evidence (data)?What do you do with the evidence (data)?What do you do with the evidence (data)?

1717

NOW THAT YOU CAN PICTURETHE WHOLE PROCESS,

LET’S QUICKLY CONSIDEREACH ELEMENT IN THE ASSESSMENT PROCESS

NOW THAT YOU CAN PICTURETHE WHOLE PROCESS,

LET’S QUICKLY CONSIDEREACH ELEMENT IN THE ASSESSMENT PROCESS

1818

Mission StatementMission Statement

An adequate An adequate mission statementmission statement clearly identifies 1) clearly identifies 1) what your program is, 2) what your program does (why what your program is, 2) what your program does (why you exist), 3) how you do it, and 4) who your program you exist), 3) how you do it, and 4) who your program serves (not necessarily in that order)... it is serves (not necessarily in that order)... it is notnot a a idealistic “idealistic “visionvision” statement, which delineates what you ” statement, which delineates what you want to accomplish or become in the want to accomplish or become in the futurefuture

Usually accomplished once; updated from time to time Usually accomplished once; updated from time to time based on assessment data. The program mission based on assessment data. The program mission must support the CBU mission, priorities, and must support the CBU mission, priorities, and University Student OutcomesUniversity Student Outcomes

An adequate An adequate mission statementmission statement clearly identifies 1) clearly identifies 1) what your program is, 2) what your program does (why what your program is, 2) what your program does (why you exist), 3) how you do it, and 4) who your program you exist), 3) how you do it, and 4) who your program serves (not necessarily in that order)... it is serves (not necessarily in that order)... it is notnot a a idealistic “idealistic “visionvision” statement, which delineates what you ” statement, which delineates what you want to accomplish or become in the want to accomplish or become in the futurefuture

Usually accomplished once; updated from time to time Usually accomplished once; updated from time to time based on assessment data. The program mission based on assessment data. The program mission must support the CBU mission, priorities, and must support the CBU mission, priorities, and University Student OutcomesUniversity Student Outcomes

1919

1.1. Write Write student learning (including faith integration) outcomesstudent learning (including faith integration) outcomes2.2. Write institutional capacity Write institutional capacity outcomes (optional)outcomes (optional)3.3. Outcomes-goals are mappedOutcomes-goals are mapped

1.1. Write Write student learning (including faith integration) outcomesstudent learning (including faith integration) outcomes2.2. Write institutional capacity Write institutional capacity outcomes (optional)outcomes (optional)3.3. Outcomes-goals are mappedOutcomes-goals are mapped

Each program may develop as many outcome Each program may develop as many outcome statements as deemed necessary, but be realistic statements as deemed necessary, but be realistic and pragmatic. What can you and pragmatic. What can you reallyreally accomplish? accomplish?

Each program may develop as many outcome Each program may develop as many outcome statements as deemed necessary, but be realistic statements as deemed necessary, but be realistic and pragmatic. What can you and pragmatic. What can you reallyreally accomplish? accomplish?

OUTCOMESOUTCOMESStep TwoStep Two involves involves threethree activities: activities:

OUTCOMESOUTCOMESStep TwoStep Two involves involves threethree activities: activities:

Outcomes are used in formulating your program’s Outcomes are used in formulating your program’s Overall Assessment Plan (OPlan) and Overall Assessment Plan (OPlan) and subsequent Yearly Assessment Plans (APlan)subsequent Yearly Assessment Plans (APlan)

2020

Academic Academic assessment plans include student learning assessment plans include student learning outcomes (including faith integration) and optional outcomes (including faith integration) and optional institutional capacity outcomesinstitutional capacity outcomes

Student ServicesStudent Services assessment plans include mainly assessment plans include mainly institutional capacity outcomes (administrative goals), institutional capacity outcomes (administrative goals), but in some cases also include student learning but in some cases also include student learning outcomesoutcomes

OUTCOMESOUTCOMESOUTCOMESOUTCOMES

2121

Overall Assessment Plan - “OPlan”Overall Assessment Plan - “OPlan”

The assessment “Big Picture” or “catalog” listing basic The assessment “Big Picture” or “catalog” listing basic information about the program’s outcomes and related information about the program’s outcomes and related general assessment informationgeneral assessment information

As descriptive as necessary to identify the various As descriptive as necessary to identify the various elements in the overall plan, but exacting specificity is elements in the overall plan, but exacting specificity is not necessary because such information is reserved for not necessary because such information is reserved for the yearly assessment planthe yearly assessment plan

2222

Overall Assessment Plan - “OPlan”Overall Assessment Plan - “OPlan”

The Program’s Overall Assessment Plan identifies:The Program’s Overall Assessment Plan identifies:

► Where each outcome is addressedWhere each outcome is addressed— specific courses, — specific courses, services, internships, service projects, work services, internships, service projects, work experiences, independent studies, etc. (based on the experiences, independent studies, etc. (based on the previously completed “map”)previously completed “map”)

► Proposed assessmentProposed assessment methodmethodss (the ideal is 2 methods (the ideal is 2 methods per outcome)per outcome)

► Criteria for Criteria for eacheach assessment method assessment method— the desired, — the desired, intended performance standard, the level of acceptable intended performance standard, the level of acceptable performance or accomplishmentperformance or accomplishment

► Assessment frequencyAssessment frequency— how often assessed; yearly, — how often assessed; yearly, every other year, etc. every other year, etc.

2323

Measuring Educational EffectivenessMeasuring Educational EffectivenessMeasuring Educational EffectivenessMeasuring Educational Effectiveness

Assessment methods must be appropriate to Assessment methods must be appropriate to the academic discipline and level or student the academic discipline and level or student service area; select and use service area; select and use either or botheither or both

quantitativequantitative or or qualitative qualitative strategies for strategies for measuring stated outcomesmeasuring stated outcomes

There are many different assessment strategies There are many different assessment strategies and methods. Select at least one option for and methods. Select at least one option for

each outcome you intend to measureeach outcome you intend to measure

2424

Some Assessment Every YearSome Assessment Every Year

The goal is to accomplish The goal is to accomplish somesome assessment each assessment each year ... year ... accumulating evidence overaccumulating evidence over timetime, building a , building a weight of evidence. It is not necessary or possible to weight of evidence. It is not necessary or possible to assess everything every yearassess everything every year

Assessment must be realistic and manageableAssessment must be realistic and manageable

So, find what works and do a little each year So, find what works and do a little each year

2525

Annual Assessment Plan - “APlan”Annual Assessment Plan - “APlan”

• Identify two or three (number can vary) student learning Identify two or three (number can vary) student learning outcomes and perhaps optional institutional capacity outcomes and perhaps optional institutional capacity outcomes to assess during the up-coming yearoutcomes to assess during the up-coming year

• Focusing on just a few outcomes each year adds up over Focusing on just a few outcomes each year adds up over time and results in all the outcomes being assessed two time and results in all the outcomes being assessed two or three times within a ten-year periodor three times within a ten-year period

• Annual Assessment Plans are developed each year– for Annual Assessment Plans are developed each year– for the subsequent fiscal year– after the Yearly Assessment the subsequent fiscal year– after the Yearly Assessment Report is completedReport is completed

• In some limited cases, particular outcomes may require In some limited cases, particular outcomes may require assessment each year assessment each year

2626

Annual Assessment Plan - “APlan”Annual Assessment Plan - “APlan”

1) WHAT assessment method(s)—identify the exact assessment method(s) to be used

2) WHEN the assessment takes place—date(s), time(s)

3) HOW the assessment method is accomplished—the specific steps or procedures, including any necessary materials

4) WHO is responsible to complete the assessment—person or persons who are tasked with actually implementing the assessment process and collecting the assessment data

Identify four elements for each selected student learning or optional institutional capacity outcome:

2727

Time for action. . . During the school year or at During the school year or at other designated times,other designated times, collect the desired assessment data (evidence)

Time for action. . . During the school year or at During the school year or at other designated times,other designated times, collect the desired assessment data (evidence)

Follow your Annual Assessment Plan (APlan), but make whatever adjustments are needed along the way to insure collecting the data you need

Follow your Annual Assessment Plan (APlan), but make whatever adjustments are needed along the way to insure collecting the data you need

DO ASSESSMENTDO ASSESSMENT

2828

Doing AssessmentDoing Assessment

This is the “This is the “so whatso what” process within assessment It is ” process within assessment It is necessarynecessary to take action based on the resulting to take action based on the resulting

assessment informationassessment information

This is the “This is the “so whatso what” process within assessment It is ” process within assessment It is necessarynecessary to take action based on the resulting to take action based on the resulting

assessment informationassessment information

Once the desired data (evidence) is collected and Once the desired data (evidence) is collected and analyzed, make decisions (evaluate) supported by analyzed, make decisions (evaluate) supported by data, which lead to making adjustments or changes data, which lead to making adjustments or changes

in the program, curriculum, processes, etc.in the program, curriculum, processes, etc.

Once the desired data (evidence) is collected and Once the desired data (evidence) is collected and analyzed, make decisions (evaluate) supported by analyzed, make decisions (evaluate) supported by data, which lead to making adjustments or changes data, which lead to making adjustments or changes

in the program, curriculum, processes, etc.in the program, curriculum, processes, etc.

2929

Yearly Assessment ReportYearly Assessment Report

Every year after the annual assessment cycle is Every year after the annual assessment cycle is completed, it is time to file a short report, via LiveText, completed, it is time to file a short report, via LiveText, outlining the results and any actions taken or planned as outlining the results and any actions taken or planned as a result of the assessment dataa result of the assessment data

Assessment reports are due Assessment reports are due July 31July 31, annually, and are , annually, and are based on the preceding fiscal year— July 1 to June 30, based on the preceding fiscal year— July 1 to June 30, including summer sessionsincluding summer sessions

Once the report is submitted, it is time to develop next Once the report is submitted, it is time to develop next year’s Annual Assessment Planyear’s Annual Assessment Plan

3030

Yearly Assessment ReportYearly Assessment Report

(1)(1) List the specific outcome List the specific outcome

(2)(2) Briefly describe the assessment method(s) usedBriefly describe the assessment method(s) used

(3)(3) Summarize the evidence collected—what you found Summarize the evidence collected—what you found out out

(4)(4) Describe the actions taken or planned based on the Describe the actions taken or planned based on the assessment evidence assessment evidence

(5)(5) (Optional) Cite any lessons learned(Optional) Cite any lessons learned

For each student learning and institutional capacity outcome assessed during the past fiscal year, the assessment report must contain four or five sections:

3131

State your program’s mission

Write student learning (including faith integration) and institutional capacity outcomes; “map” your outcomes

Explore and select assessment methods

Develop an overall assessment plan (OPlan)

Each year write an annual implementation plan (APlan)

Do assessment

Collect and analyze data (evidence)

Use data to make identified adjustment, improvements, etc.

Each year report assessment actions and results (Yearly Report)

Implement improvements

Summary: BASIC Steps in Assessment

3232

LiveText is the administrative software Used to Facilitate the

CBU Assessment Process

LiveText is the administrative software Used to Facilitate the

CBU Assessment Process

LiveText is Web-based portfolio software

All academic programs and student service departments are required to maintain a assessment portfolio in LiveText

The program’s assessment portfolio requires specific assessment documents (OPlan, APlans, etc.), but faculty/staff enjoy wide latitude in writing appropriate outcomes and selecting assessment methods

LiveText is Web-based portfolio software

All academic programs and student service departments are required to maintain a assessment portfolio in LiveText

The program’s assessment portfolio requires specific assessment documents (OPlan, APlans, etc.), but faculty/staff enjoy wide latitude in writing appropriate outcomes and selecting assessment methods

3333

Using Assessment ResultsUsing Assessment Results

At the appropriate times, use the collected assessment information to write...

At the appropriate times, use the collected assessment information to write...

Yearly Assessment ReportsYearly Assessment Reports CBU Program ReviewsCBU Program Reviews WASC Self-Study ReportsWASC Self-Study Reports Professional Accreditation ReportsProfessional Accreditation Reports Other documents requiring assessment Other documents requiring assessment

data (e.g., grant applications)data (e.g., grant applications)

REMEMBER:REMEMBER: Must act upon the assessment dataMust act upon the assessment data Assessment is on-going and never endsAssessment is on-going and never ends

3434

Office of Institutional Research Office of Institutional Research Planning, and Assessment (OIRPA)Planning, and Assessment (OIRPA)

► The OIRPA exists to help academic and student life departments succeed in their assessment activities

► The OIRPA does not do evaluation, but assists programs, as needed, to collect evidence (data) so they can evaluate their own programs

► The OIRPA does not stipulate a program’s outcomes or assessment methods, this is a programmatic prerogative

► The OIRPA depends on the programs’ assessment activities as a basis for making statements about assessment at CBU as a whole

► The OIRPA exists to help academic and student life departments succeed in their assessment activities

► The OIRPA does not do evaluation, but assists programs, as needed, to collect evidence (data) so they can evaluate their own programs

► The OIRPA does not stipulate a program’s outcomes or assessment methods, this is a programmatic prerogative

► The OIRPA depends on the programs’ assessment activities as a basis for making statements about assessment at CBU as a whole

3535

Assessment AssistanceAssessment Assistance

Who to go to for help: Who to go to for help:

Tools provided to help you:Tools provided to help you:

1.1. The Assessment Coordinator in your The Assessment Coordinator in your School or College or Student Services School or College or Student Services 2.2. OIRPA OIRPA

• Assessment trainingAssessment training• Assessment “how-to” manualsAssessment “how-to” manuals• Public Folders documentsPublic Folders documents• LiveText software and trainingLiveText software and training• Etc.Etc.

3636

Office of Institutional Research Office of Institutional Research Planning, and Assessment Planning, and Assessment

(OIRPA)(OIRPA)

Personnel working with assessment:Personnel working with assessment:

Dr. Neal McBride, Associate ProvostDr. Neal McBride, Associate Provost

Phil Martinez, Director of AssessmentPhil Martinez, Director of Assessment

Office Location: James 103Office Location: James 103

Telephone: 951-343-4346Telephone: 951-343-4346

Our job is to help YOU succeed in doing assessment

3737

Programss with professional accreditation find Programss with professional accreditation find that 80-95% of the assessment activities that 80-95% of the assessment activities

necessary for their professional accreditation necessary for their professional accreditation are also satisfactory for CBU and WASC are also satisfactory for CBU and WASC

assessment expectations, with only minor assessment expectations, with only minor editing required when reporting in LiveTextediting required when reporting in LiveText

3838

The Assessment ProcessThe Assessment Process