ANALYZING THE CONTENT OF CAAP

Learning College ConferenceFebruary 26-27, 2009

COLLEGIATE ASSESSMENT OF ACADEMIC PROFICIENCY (CAAP)

ACT product Use four modules college wide Each region gives only 2 of the 4 Rotation schedule ensures that regions

administer a different pair each academic year Required of all associate degree graduates Administered in capstone courses

PURPOSE OF CAAP

To measure 4 of the College’s 8 general education outcomes

Demonstrate critical and creative thinking. Apply basic scientific concepts in a variety of

settings. Communicate effectively in written, oral and

symbolic forms. Exhibit quantitative literacy.

PURPOSE OF CAAP

To identify areas of strength and weakness as compared to graduates of other public community colleges

Results reviewed by curriculum committees, regional academic officers, central academic affairs staff

To inform changes designed to improve student learning

RESULTS CURRENTLY REPORTED

Descriptive statistics Mean scores by module Mean scores by subcategory Percentile ranks Comparison of mean scores to national norms Level of effort student reported giving the

module Mean scores and level of effort by region

CAAP CONTENT ANALYSIS REPORTS

Enable analysis of specific content areas: Writing Skills

Punctuation Basic Grammar and Usage Sentence Structure Strategy Organization Style

Mathematics Prealgebra Elementary Algebra Intermediate Algebra Coordinate Geometry College Algebra Trigonometry

CAAP CONTENT ANALYSIS REPORTS

Science Analysis Generalization Understanding

Critical Thinking Analysis of Arguments Evaluation of Arguments Extension of Arguments

CAAP CONTENT ANALYSIS REPORTS

ACT’S PROCESS

Students (ITCC and nationally) are classified into proficiency groups Top 25% Middle 50% Bottom 25%

For each student, percentage of items answered correctly is calculated

For each proficiency group, students’ percent-correct scores are averaged and displayed using bar graphs

WRITING SKILLS: PUNCTUATION

http://www.act.org/caap/tests/writing.html

COMPARISONS AND DIFFERENCES

Reports differences in percentages between Ivy Tech and national groups <5% difference considered negligible 5%-10% difference considered moderate >10% difference considered substantial

Negative differences indicate areas where Ivy Tech students “had more difficulty with content category items than did the normative group”

Positive differences indicate that Ivy Tech students found items easier than did the norm

WRITING SKILL COMPARISON

N=2627

GROUP TASKS – 15 MINUTES

Divide into teams and identify a “reporter” Take results of one module Discuss and determine the following:

In relationship to students at other community colleges, where are our students’ strengths? ….where are their weaknesses?

What should the College do to improve students’ ability in the content area being assessed?

What can your program do to improve students’ ability in the content area being assessed?

REPORTS

In relationship to students at other community colleges, where are our students’ strengths? ….where are their weaknesses?

What should the College do to improve students’ ability in the content area being assessed?

What can your program do to improve students’ ability in the content area being assessed?

Cherry Kay SmithExecutive Director of Academic Policy and Assessment

317.916.7810csmith@ivytech.edu

ANALYZING THE CONTENT OF CAAP