© 2012 by act, inc. all rights reserved. act — cognitive and non- cognitive factors that...
Post on 14-Dec-2015
212 Views
Preview:
TRANSCRIPT
© 2012 by ACT, Inc. All rights reserved.
ACT — Cognitive and ACT — Cognitive and Non-Cognitive Factors Non-Cognitive Factors
that Determine Student that Determine Student College and Career College and Career
ReadinessReadinessApril Hansen, Postsecondary Director
George Schlott, Senior Consultant for Program Solutions
ACT, Inc. – Midwest Region-Lincolnshire IL
What is College and Career Readiness?
Ensuring that all students, graduating from high school, have acquired the skills in English,
mathematics, reading, and science that they need to be ready for entry-level college courses without
remediation
A Question for AllA Question for All
Planning for School
Improvement
Improving Course Rigor
Measuring Student Progress TowardCollege and Career Readiness
Research
EXPLORE
8th & 9th grade
curriculum-based
educational and career planning program
PLAN
10th grade curriculum-
based educational and career planning program
The ACT
11th & 12th grade
curriculum-based
measurement for
learning outcomes
ENGAGE
Middle and High School assessment
that measures factors of academic success
QualityCore
Research-driven
solutions for strengthening curriculum & instruction
CoreWork Diagnostics
Online service to diagnose and
improve content and practice areas
ACT College Readiness Standards
ACT College Readiness Benchmarks ACT Core Course
Standards
WorkKeys
Job skill assessments and portable credentials necessary for career
pathways
College & Career Readiness Information SystemCollege & Career Readiness Information System
Core Practice Audit
Online service to diagnose and
improve content and practice areas
ACT College & Career Readiness SystemACT College & Career Readiness System
Through collaborative research with postsecondary institutions nationwide, ACT has established the following college readiness
benchmarks*:
* Minimum score needed on an ACT subject-area test to indicate a 50% chance of obtaining a B or higher or a 75% chance of obtaining a C or higher in the corresponding first-year credit-bearing
college course.
Test College Course ACT
English English Composition 18
Math College Algebra 22
Reading Social Science 21
Science Biology 24
Test College Course EXPLORE 8 EXPLORE 9 PLAN 10 ACT COMPASS
English English Composition
Math College Algebra
Reading
Social Science
Science Biology
13
17
15
20
14
18
16
20
15
19
17
21
18
22
21
24
69
65
88
NA
ACT College Readiness BenchmarksACT College Readiness Benchmarks
Students who are college/career ready when they leave high school have a significantly higher likelihood of: – Enrolling in a postsecondary program,– Enrolling in credit bearing courses without the
need for remediation,– Succeeding in entry level postsecondary
course work,– Persisting in their postsecondary education, – Completing a postsecondary degree or
training program, and – Entering the job market with significantly
higher lifetime earning potential.Regardless of ethnicity and SES
Value of College ReadinessValue of College Readiness
© 2012 by ACT, Inc. All rights reserved. 6
• Many students are not prepared to meet the hurdles they face throughout the academic pipeline, and they don’t persist and succeed.
NCES 2010
• What can we do to improve student success and persistence?
Enter 9th grade
HS Graduat
e
Enter College
College Graduate
(Bachelor’s)
~100%
96.9% 74.9% 52.5% 29.0%
Leaky Educational Pipeline
Leaky Educational PipelineLeaky Educational Pipeline
Pop Quiz
What’s the number of American high school students who drop out of school, every day, bored, frustrated, or so far behind that they’ve given up?
6,000
Leaky Educational PipelineLeaky Educational Pipeline
Two-year schools -- 54 % Four-year private colleges -- 73%Four-year public -- 71% *
*Wes Habley, ACT, January 2009
By 2020: Cost of the average private--$60-70,000 per year Cost of the average public--$17,500-27,500 per year
In 2008:Out of approx. 1.8 million first year college students, 450,000 did not return to
the college or university where they began their college career.* That’s 25%.*Bryan Matthews, Retention Matters, Inside Higher Ed, Nov. 2, 2009
Attrition Costs MoneyAttrition Costs Money
If you have an incoming class of 1000, and lose even 10% of the class (100 students), that’s
100 x $30,000 = $3 million $3 million x 4 years = $12 million
Retaining students allows you to maintain revenue streams while keeping tuition increases down.
It’s the low hanging fruit.
Do the Math!Do the Math!
evaluate students’ personal, behavioral, and academic skills critical to high school and college achievement
determine their levels of academic risk
apply specific interventions to help them persist in their studies and achieve academic success
identify student strengths and areas for improvement in student motivation, social engagement, and self-regulation
predict college retention for each incoming freshman
What if you had an assessment that would:What if you had an assessment that would:
Research is the Foundation
1. The strongest predictors of college persistence and degree completion are: prior academic achievement and course selection (rigorous high school classes).
2. Prior academic achievement and cognitive ability surpass all other factors in their influence on student performance.
3. Non-academic factors can influence academic performance, retention and persistence, but cannot substitute for it.
Grades 6-9Grades 10-12
College Teacher
You Do!You Do!
© 2012 by ACT, Inc. All rights reserved. 14
• Create a comprehensive assessment system that predicts success in education and work
• Tailor assessments for critical transition points and context
• Measure characteristics amenable to change
• Help educators connect students’ needs to interventions
The ChallengesThe Challenges
© 2012 by ACT, Inc. All rights reserved. 15
Achievement
GradesCollege Work6 7 8 9 10 11 12
EXPLORE ® PLAN ® ACT ® COMPASS ®
ENGAGE and ENGAGE Teacher EditionENGAGEBehavior
WorkKeys ®
Talent
ACT Interest Inventory & Map of College MajorsCareer Fit
ACT’s Academic Achievement, Behavior & ACT’s Academic Achievement, Behavior & Career Planning SolutionsCareer Planning Solutions
© 2012 by ACT, Inc. All rights reserved. 16
• Students’ personal characteristics and psychosocial development influence their ability to stay in school and be successful
• ACT research shows that psychosocial/behavioral development can be structured into 3 broad domains:– Motivation – Social Engagement – Self-Regulation
• ACT’s academic behavior assessments measure these domains
ACT’s Academic Behavior AssessmentsACT’s Academic Behavior Assessments
© 2012 by ACT, Inc. All rights reserved. 17
Student Grade Levels
6 7 8 9 10 11 12
ENGAGEGrades 6-9
ENGAGE Grades 10-12
ENGAGECollege
ENGAGE Teacher Ed. Grades 6-9
ENGAGE TeacherEd. Grades 10-12
College
Student reported(surveys)
Teacher reported(teacher ratings)
ACT’s Academic Behavior AssessmentsACT’s Academic Behavior Assessments
© 2012 by ACT, Inc. All rights reserved. 18
• Surveys (Student reported)– ENGAGE Grades 6-9 (typically in 6th & 9th
grade)– ENGAGE Grades 10-12 (typically in 11th grade)– ENGAGE College (typically early in 1st
semester)
• Behavioral Measures (Teacher reported)– ENGAGE Teacher Edition for Grades 6-9 and
10-12 (completed by teachers 2-3x per year for each student)
COMING SOON!
ACT’s Academic Behavior AssessmentsACT’s Academic Behavior Assessments
© 2012 by ACT, Inc. All rights reserved. 19
Domain ENGAGE Grades 6-9ENGAGE Grades 10-12 & College
Motivation (Getting work done)
Academic Discipline Commitment to School Optimism
Academic Discipline Commitment to College Goal Striving General Determination Study Skills Communication Skills
Social Engagement(Getting along)
Family Attitude toward Education
Family Involvement Relationships w/ School
Personnel School Safety Climate
Social Activity Social Connection
Self-Regulation(Keeping your cool)
Managing Feelings Thinking Before Acting Orderly Conduct
Academic Self-Confidence Steadiness
ACT’s Academic Behavior AssessmentsACT’s Academic Behavior Assessments
© 2012 by ACT, Inc. All rights reserved. 22
• Measures students’ perceptions of themselves, their families’ commitment to education, school-related factors, and important behavioral data
• Developed to predict academic success and high school graduation
• Low stakes, self-report instrument, 4th-grade reading level
• 10 scales• Online administration
ENGAGE: Grades 6-9ENGAGE: Grades 6-9
© 2012 by ACT, Inc. All rights reserved. 23
• Measures students’ psychosocial attributes, determines level of risk, and helps identify interventions to promote successful transition to postsecondary studies
• Developed to predict academic success and retention in the first year of college
• Low stakes, self-report instrument, 6th-grade reading level
• 10 scales• Online administration
ENGAGE 10-12 & CollegeENGAGE 10-12 & College
© 2012 by ACT, Inc. All rights reserved. 24
• Academic Success and Graduation/Retention Indices
• Student profiles can be used to:– Identify relative strengths and needs– Inform student advising – Match students’ needs to interventions
• Aggregate reports • Roster reports• All reports available online 24/7
ENGAGE: ReportsENGAGE: Reports
© 2012 by ACT, Inc. All rights reserved. 25
Profile of scores
Success Indices(only on Advisor Report)
Interpretive feedback, sorted from strengths to needs
ENGAGE: Sample Advisor ReportENGAGE: Sample Advisor Report
© 2012 by ACT, Inc. All rights reserved. 26
ENGAGE: Sample Profile ReportENGAGE: Sample Profile Report
© 2012 by ACT, Inc. All rights reserved. 27
ENGAGE: Sample Intepretative ReportsENGAGE: Sample Intepretative Reports
© 2012 by ACT, Inc. All rights reserved. 28
ENGAGE: School/Institution Aggregate ReportENGAGE: School/Institution Aggregate Report
© 2012 by ACT, Inc. All rights reserved. 29
ENGAGE: Roster ReportENGAGE: Roster Report
© 2012 by ACT, Inc. All rights reserved. 30
• Grounded in research– Developed based on meta-analysis of predictors of
academic performance and persistence (Robbins et al., 2004)
– Validated across many studies (e.g., Allen et al., 2010; Casillas et al., in press; Robbins et al., 2006, 2008)
• Helps educators to consider the whole student– Provides profile of relative strengths and needs
• Practical and flexible– Administer online at any time– Reports available online 24/7– Identify areas of challenge for students– Create a crosswalk of ENGAGE scales to existing
services
Why ENGAGE?Why ENGAGE?
© 2012 by ACT, Inc. All rights reserved. 31
Percentage of students accurately identified
as having a 9th-grade GPA <2.0
Selection Method Hit Rate
Random 24%
EXPLORE Composite only
69%
ENGAGE Grades 6-9 only
80%
EXPLORE & ENGAGE 83%
© 2012 by ACT, Inc. All rights reserved. 32
Average Early High School GPA, by EXPLORE
and ENGAGE Grades 6-9 Scores
1.771.99
2.89
1.96
3.34
2.41
3.13
3.63
2.62
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
4.0
Bottom 25% Middle 50% Top 25%
EXPLORE Composite Score
Av
era
ge
Ea
rly
HS
GP
A
Low ENGAGE Moderate ENGAGE High ENGAGE
© 2012 by ACT, Inc. All rights reserved. 33
Percentages of Students Accurately Identified as Being at
High Risk for Dropping Out
Selection MethodHit
Rate
Random 17%
EXPLORE Composite only 37%
ENGAGE Grades 6-9 Grad. Index only
42%
EXPLORE & ENGAGE Grad. Index
52%
© 2012 by ACT, Inc. All rights reserved. 34
Persistence Rates, by EXPLORE and ENGAGE Grades 6-9 Graduation Index Levels
48
62
72
89
9692
9698
69
0
20
40
60
80
100
Bottom 25% Middle 50% Top 25%
EXPLORE Composite Score
Pe
rsis
ten
ce
Ra
te
Low Graduation Index Moderate Graduation Index High Graduation Index
© 2012 by ACT, Inc. All rights reserved. 35
Use of ENGAGE College to Identify Students Academically at Risk
Note. Students scoring in the bottom 5% of each of these populations were flagged.
Accuracy of Identification
Selection Method
Drop OutAcademic Difficulty
Random 10% 20%
ENGAGE Success Indices
24% 46%
© 2012 by ACT, Inc. All rights reserved. 36
• Designed to assess and monitor students’ development
• Students are rated by teachers who know them well
• Requires each student to be rated by at least one teacher
• Used to 2-3 times per year• Scales are “anchored” with behavioral
statements to enhance accuracy and reliability
• Same dimensions for Grades 6-9 and 10-12, but rating scales are made up of different items.
ENGAGE: TeacherENGAGE: Teacher
© 2012 by ACT, Inc. All rights reserved. 37
Domain Scales
Motivation
(Getting work done)
Initiative Planning & Organizing Sustained Effort Performance
Social Engagement
(Getting along)
Communication Working with Others
Self-Regulation
(Keeping your cool)
Managing Feelings Orderly Conduct
ENGAGE: TeacherENGAGE: Teacher
© 2012 by ACT, Inc. All rights reserved. 38
Combining Assessment and Interventions
to Promote Student Success
ENGAGE
ENGAGETeacher Edition
Support Resource
s &
Interventions
Reduced Risk
Continued Evaluation & Feedback
© 2012 by ACT, Inc. All rights reserved. 39
Example Crosswalk between Interventions and
Areas of Need Interventions
Areas of Need
Goal Setting& Time
Management
Mentoring
Cross-Age Tutoring
Community Service
Group Work &
Communication Skills
Self-Confidenc
e Workshop
How to Follow Rules
Motivation x x x x Social
Engagement
x x x x
Self-Regulation x x x
© 2012 by ACT, Inc. All rights reserved. 4040
Behavioral Education Solutions
• Grade 6-12 curriculum– Units designed to help students build
behavioral skills– Based on carefully developed Behavioral
Learning Objectives– Fully developed and structured lesson plans
and activities
• ENGAGE Tool Shop (Postsecondary) – Several tools available for each ENGAGE scales
to address students’ needs– Can be completed by students on their own or
as part of FYE courses or other group settings
COMING SOON!
© 2012 by ACT, Inc. All rights reserved. 41
ENGAGE Tool Shop (College)
© 2012 by ACT, Inc. All rights reserved. 42
Intervention Research
• University of North Texas– Large university (36,000 students)– Administers ENGAGE to all incoming 1st-year
students– Identify at-risk students and follow up with
immediate individualized intervention• Northern Arizona University
– 4-year public institution serving 13,000 students
– Use ENGAGE to guide intrusive interventions with those students who need the most help
© 2012 by ACT, Inc. All rights reserved. 43
UNT Face-to-Face Meeting
• Advisor or student affairs staff meet with student during first 5-6 weeks of semester
• Present ENGAGE results by focusing first on strengths, then needs
• Focus resources to aid student in: – building on one strength and addressing one
area for improvement– using crosswalk of ENGAGE to campus
resources
© 2012 by ACT, Inc. All rights reserved. 44
Sample Crosswalk of Resources
SRI Scale Definition ResourcesAcademic Discipline
Effort put into school work and the degree to which students see themselves as hardworking and conscientious.
• Learning Center• Office of Exploring Majors• UNT Math Tutor Lab• UNT Student Writing Lab
Social Connection Feelings of connection and involvement with school and community.
• Off-Campus Student Services
• Transfer Center• Residence Life• Recreation Center
Academic Self-Confidence
Belief in ability to perform well in school.
• Learning Center• Counseling and Testing• Office of Exploring Majors
© 2012 by ACT, Inc. All rights reserved. 45
www.act.org/engage/index.html
© 2012 by ACT, Inc. All rights reserved. 46
www.act.org/engage/index.html
For each version of ENGAGE, website contains:– Features and Benefits– Testing Timeline– Program Materials– Research– Results and Reports
top related