comprehensive assessment reports

1

Fred Trapp, Ph.D.Administrative Dean, Institutional Research/Academic Services (Retired)

Long Beach City CollegeCambridge West Partnership, LLC

Robert Pacheco, Ed.D.Dean Of Research, Development and Planning

Barstow College

Comprehensive Assessment Reports

2

Outcomes for the Session

The participant will be able to

Describe the comprehensive assessment report concept.

Critique selected best practice examples from other colleges.

Discuss how the comprehensive report idea can be used at your college as part of the institution’s learning, planning and/or resource allocation processes.

Note: Wiki, Cloud and Video

3

Curriculum Map3

4

ACCJC Institutional Effectiveness Rubric (2011)

Part III Student Learning Outcomes Proficiency

Comprehensive assessment reports exist, are completed and updated on a regular basis.

5

What Did We Look At, With Whom Did We Consult?

ACCJC. Institutional Effectiveness Rubric

ACCJC. 2002 StandardsACCJC. ThemesACCJC. Guide to Evaluating InstitutionsProfessional literatureEfforts of national groups/institutesInstitutional web sites, listservs and colleagues

6

Guiding Questions

How can the report writing experience: Help faculty explore the student learning process?

Determine the extent to which the curriculum is working?

Inform the decisions as to where time, energy and/or money be allocated for continuous improvement in learning?

Stimulate dialogue about results to gain broader institutional learning experiences?

Help meet our quality assurance pledge to the community?

7

Illustration Selections

Council for Higher Education Accreditation (CHEA) Annual Award for Outstanding Institutional Practice in Student Learning Outcomes Demonstrated commitment to & developed highly effective practice to use SLO assessment

Willingness to share the practices they developed Selection committee Selection criteria

Articulation & evidence of outcomes Success with regard to outcomes Information to the public about outcomes Using outcomes for improvement

8

Illustration selections (continued)

Cited by scholars and peers Schools with assessment work cited in scholarly books and articles

Schools with assessment work selected for presentation at national conferences

Web presentations publicly available for you to eavesdrop upon

9

What Might Be Included?

1. Assessment focus-course, program, general ed., etc.

2. What outcomes were assessed?3. Which, how and when were students

assessed?4. What were the results?5. Who reviewed the results, made sense of

the them and what conclusions were reached?

6. What are the implications for practice and/or policy or future assessment work?

10

CC of Baltimore County (MD)Course-level Reporting

Middle States Commission on Higher Education

Community College Futures Assembly, Bellwether Award, 2008 Instructional Programs & Services for High Impact Course Level Assessment

CHEA award winner, 2006 Institutional Progress in Student Learning Outcomes

National Council on Student Development (NCSD) Exemplary Practice Award Winner

11

CC of Baltimore County (MD)Course-level Reporting

Projects are at least three semesters long Individual and high-impact courses (all sections) included

Project proposal by a faculty group Measurable objectives External review & approval in selecting methods/instrument & analyzing results. Benchmarking should be included if possible.

Controls and sample size considered. Course improvements based on data analysis Reassessment expected Results/report shared across the college and web posted

12

CC of Baltimore County (MD)

Example report CHEM 108

An initial “failure” turned to success and collaboration with a four-year school

Activity Read the report In small groups respond to the discussion questions Questions 2 and 3

Report out to larger group

13

14

Hocking College (OH)Program-level Reporting

North Central Association of Colleges & Schools, Higher Learning Commission

A career & technical education institution

CHEA award winner, 2008 Institutional Progress in Student Learning Outcomes

15

Hocking College (OH)

Learning outcomes data collected in a student E-portfolio Direct internal and external evidence (1 to 10 measures)

Indirect evidence (1 to 4 measures)

Evidence drawn from samples of student work for faculty to apply an agreed upon holistic rubric Eight general education outcomes (student success skills)

Discipline-specific exit competencies or outcomes

Hocking College Triangulation16

Hocking College (OH)Program-level Report

Example report and analysis Practical Nursing Technology

Activity Read the report In small groups respond to the discussion questions

Questions 3, 4, and 5 Report out to larger group

17

18

19

20

21

Mesa College (AZ)General-Education Reports

North Central Association of Colleges & Schools, Higher Learning Commission

A large comprehensive community collegeCHEA Award winner, 2007

Institutional progress in Student Learning Outcomes

22

Mesa Community College (AZ)

General education studies completed 2007-08; 2005-06 Numeracy Scientific inquiry Problem solving/critical thinking (2008-09) Global Awareness (2008-09 pilot test) Information literacy Workplace skills (CTE) (2007-8; 2009-10)

General education studies completed 2006-07; 2004-05 Arts & humanities Cultural diversity Oral communication Written communication

23

Mesa College (AZ)General Education Reports

Annual Report 2008-09 example General Education section only

Activity Read the report In small groups respond to the discussion questions

Report out to larger group

24

25

26

From page 14

27

From page 15

28

29

Authorship

Course-level, program-level & general education Teaching faculty study team with technical assistance from institutional research or assessment committee

No “lone ranger” authors

Institutional summary Academic administrator with assistance from

Learning outcomes coordinator or assessment committee Compilation of work accomplished in one or two academic years across the institution

Your Uses

How do you see your school using annual comprehensive reports?

30

See last four pages in the Handout and Discussion Questions packetNorthern Arizona U.- Assessment Report Evaluation RubricU. of Illinois Urbana-Champaign- Reported Unit Improvements from Outcomes Assessments 2000-2008

31

Putting the Reports to “Work”

To all affected participants & on at least the intranet

Campus committees Curriculum, assessment, resource allocation group, unit (department) leadership, general academic and college leadership

Campus fairs, brown-bag lunches, poster sessions for information sharing

Faculty professional development programsAccreditation self-study committee work groupsUnit plans, program reviews, requests for resources

College web site for the public

32

Reports & Institutional Learning

The assessment data reflection activity sense-making process = a faculty learning experience

Linking results to future interventions = a learning experience

Using results to inform an intervention, then reassess = a learning experience (accomplished one or more terms later)

Reference for future assessment work and other groups on campus

33

Reports as Institutional Learning& Resource Allocation

Hocking College (OH) Annual summary

Improvements in the program in the previous year brought on by study of assessment results

Expenditures of time, money & materials for the assessment program

Requests for assistance in implementing assessment Recommendations for altering the institution’s assessment process

Mega faculty learning insight transition from evaluating individual students to assessing groups of students & the curriculum experience

34

Reports as Institutional Learning & Resource Allocation

Community College of Baltimore County (MD) Learning Outcomes Assessment Advisory Board

Links findings in assessment reports to other college-wide initiatives and professional development opportunities

Use of assessment processes and (findings) results Challenged faculty to reexamine prompts used in assessment Greater clarity of written prompt See how to support program goals

Common assignment options and common rubric increases faculty understanding and buy-in

Public web page enhances communication and accessibility to information

35

Reports as Academic Quality& Resource Allocation

Mesa College (AZ)Results Outreach Committee

Promotes use of outcomes data in relation to faculty development, pedagogy and academic climate

Groups of faculty offer a proposal for summer or academic year work above the course level

Resulting report placed on the web and used for campus discussion and action

36

Report as Quality Assurance

Promising Vehicles for Expanding Information to the PublicBrief narrative report from annual assessment reports

Simple statistical reports on learning outcomes or surveys

Best practices stories supported by assessment

Peter Ewell. Accreditation & the Provision of Additional Information to the Public about Institutional and Program Performance, CHEA, May 2004

37

38

39

Where Can I Go?Resources

Cloudhttp://www.filesanywhere.com/fs/v.aspx?v=8a6d6986596476b7a5ae

Wiki www.comprehensiveassessmentreports.wikispaces.com

Video

Contacts & Questions

Robert Pacheco (Barstow College)Rpacheco@Barstow.edu

Fred Trapp (Cambridge West Partnership)fredtrapp@gmail.com

Questions and Comments

42