Assessing SUNY General EducationLearning Outcomes with

April 27, 2005

Presenters

David ChadimaConsultant, Postsecondary Assessment Services, ACT

Robert ZiomekDirector of Education and Workforce Research Services, ACT

Frances DearingAssistant Dean of Assessment, Academic Support and Placement, Westchester County Community College

Lanette RaymondResearch Associate, Suffolk County Community College

Presentation Outline

CAAP Overview SUNY Outcomes / CAAP Test Module

Alignment & Report Customization Critical Thinking Pilot Studies

Suffolk County Community College Westchester County Community College

Q&A

What is CAAP?

Collegiate Assessment of Academic Proficiency

The standardized assessment program

from ACT that enables postsecondary

institutions to measure, evaluate, and

enhance the outcomes of their general

education programs.

CAAP Features

Six independent, 40-minute test modules

Flexible, in-class administration Comprehensive reporting at the cohort

and individual student levels Extensive user norms Linkage to ACT, COMPASS or ASSET

to measure “value added”

CAAP User Norms

to

CAAP User Norms

National norms for each test module (and subscore)

Broken out by: Institution type Year of students tested Institution ownership

SUNY system norms (2-year & 4-year if sufficient volume.

CAAP Test Modules

CAAP Test Modules

Five objective tests Writing Skills Reading Mathematics Science Critical Thinking

One writing essay test

SUNY Learning Outcomes Knowledge and Skills Areas

Critical Thinking Basic Communications (Written) Mathematics

Outcomes/Test Alignment

SUNY Outcomes CAAP Tests Critical Thinking Critical Thinking Basic Communications

(Written)

Writing Skills Writing Essay

Mathematics Mathematics

Outcomes/Test Alignment

SUNY Outcomes CAAP Tests Critical Thinking Critical Thinking Basic Communications

(Written)

Writing Skills Writing Essay

Mathematics Mathematics

Outcomes/Test Alignment

KEY ISSUE: For each test module, provide CAAP

subscores that map to SUNY learning objectives

Critical Thinking

Critical Thinking

Critical Thinking

Critical Thinking

32 items / 4 passages Subject matter: Social Sciences, Natural

Sciences, Humanities Formats:

Case Studies Dialogues Debates Editorials

Statistical Arguments Experimental Results Overlapping Position

Critical Thinking Sample

Question following passage

It is obvious from Favor's speech that she believes which of the following?

A. Most prescriptions are unnecessary.

B. Senator Counter will oppose the bill.

C. If the bill is passed into law it will greatly reduce the cost of all medical treatment.

D. If the bill is passed the average costs for treatment of minor ailments would be reduced significantly

Basic Communication (Written)

Basic Communication (Written)

Writing Essay

Two independent prompts 20 minutes each Essays scored in a holistic manner

ACT can scoreLocal faculty can score

Writing Essay Sample

Your college administration is considering whether or not there should be a physical education requirement for undergraduates. The administration has asked students for their views on the issue and has announced that its final decision will be based on how such a requirement would affect the overall educational mission of the college. Write a letter to the administration arguing whether or not there should be a physical education requirement for undergraduates at your college.

Writing Skills

72 items / 6 passages Subscores:

Usage / mechanics Rhetorical skills

Spelling, vocabulary, and rules of grammar not tested

Writing Skills Specs

Content Category % of Test # of Items

Usage/Mechanics .44 32Punctuation .08 6Grammar .11 8Sentence Structure .25 18

Rhetorical Skills .56 40Strategy .21 15Organization .14 10Style .21 15

Total 1.00 72

Writing Skills Sample

By the fourth day I had to face the truth: my body was slowly changing to becoming dough. So I tied on my running shoes and loped out to the main road in search of a five-mile route.

A. NO CHANGE

B. become

C. being

D. OMIT the underlined portion

Mathematics

Mathematics

CAAP Administration

Select test modules that match the needs of your gen ed program

2-week test window Convenient in-class administration

40 minutes per test Pencil & paper Proctored

CAAP Administration

Return materials to ACT for scoring & reporting Objective tests - ACT proprietary

scanning Essay tests - two ACT-trained scorers Three week turn-around time

CAAP Reports

Standard Reporting Package SUNY Subscore Reports

Standard Reporting Package

Includes: Institutional Summary Report Student Score Reports (2) Student Roster Report Student Certificates of Achievement Subgroup Reports (first 3 included free) Summary of Student Motivation

Data files available

SUNY Subscore Report

Subscores map to SUNY learning objectives (CT)

Groups students by top 25% middle 50% bottom 25%

Compares local student performance with state or national norms

SUNY Criteria Compliance Will it directly measure student

learning? YES

Is the measure is reliable?

Will it measure the objective it is intended to measure?

Does the plan include externally referenced norms?

Is it capable of measuring value-added?

Will the data that are reported be representative?

YES

YES

YES

YES

YES

CAAP Resources

www.act.org/caap

CAAP Resources

CAAP manuals Planning & Forms Tech Handbook Reporting &

Research Services Motivating Students

Contact:

David ChadimaACT Postsecondary Assessment Services 500 ACT Drive P.O. Box 168 Iowa City, Iowa 52243-0168Tel: (800) 294-7027Fax: (319) 337-1790E-mail: david.chadima@act.org