copyright © 2012 american institutes for research all rights reserved. casel/novo collaborating...
TRANSCRIPT
Copyright © 2012
American Institutes
for Research
All rights reserved.
CASEL/NoVo Collaborating Districts Initiative for Social and Emotional Learning:
Cross-District Evaluation Report
David Osher, Kimberly Kendziora, Lawrence B. Friedman, and the AIR CDI Evaluation Team
November 8, 2012
2
Key Questions: Overall CDI Evaluation
• What are the outcomes in districts? In schools?• How did the districts implement SEL over time?• What factors influence implementation quality?• To what extent and in what respects are
intended student-level outcomes realized?• How are realized student outcomes associated
with the school- and district-level implementation of the CDI?
3
Key Questions: Focus of this Presentation
• What is the progress of district activities and outcomes as summarized by rubric scores? 2011 and 2012 scores for Cohort 1 2012 scores for Cohort 2
• Student social and emotional competence at baseline
• Challenges and solutions• Next steps
4
Methods•Two cohorts of districts:
• Cohort 1 = Anchorage, Austin, and an R&D district, Cleveland• Cohort 2 = Chicago, Nashville, Oakland, Sacramento,
Washoe County
•District interviews 97 stakeholders across the 8 districts were interviewed
•Data examined Planning documents, communication materials, handouts,
district reports, consultant input Districtwide staff SEL surveys Student data, climate data, student social and emotional skills
ratings
•Triangulation of data District rubric scores and district reports
5
Cohort 1 Districts: Scores from 2011 and 2012
Vis
ion
Cen
tral
off
ice
expe
rtis
e
PD
pro
gram
s
Alig
n re
sour
ces
Com
mun
icat
ions
SE
L st
anda
rds
for
PK
–12
Evi
denc
e-ba
sed
prog
ram
s
Inte
grat
e S
EL
with
oth
er in
itiat
ives
Con
tinuo
us im
prov
emen
t
Pos
itive
clim
ate
Sta
keho
lder
com
mitm
ent 0
0.5
1
1.5
2
2.5
3
3.5
Rubric Scores for Cohort 1 Districts: 2011 (Planning Phase) and 2012 (Implementation Year
1)
2011
2012
6
Cohort 2 Districts: Planning Phase Rubric Scores
Nee
ds &
res
ourc
es
Vis
ion
Cen
tral
off
ice
expe
rtis
e
PD
pro
gram
s
Alig
n re
sour
ces
Com
mun
icat
ions
SE
L st
anda
rds
for
PK
–12
Evi
denc
e-ba
sed
prog
ram
s
Inte
grat
e S
EL
with
oth
er in
itiat
ives
Con
tinuo
us im
prov
emen
t
Pos
itive
clim
ate
Sta
keho
lder
com
mitm
ent
Rol
es &
res
pons
ibili
ties
0
0.5
1
1.5
2
2.5
3
2012 Rubric Scores: Cohort 2
7
Grade 7 Students’ Social and Emotional Competence: Baseline
Self-a
waren
ess
Self-m
anag
emen
t
Social
awar
enes
s
Relatio
nship
skil
ls
Resp
Decisi
on-m
aking
0%
20%
40%
60%
80%
100%
23%18% 22%
15% 17%
29%33%
33%
29%30%
28% 31% 28%38%
38%
21% 17% 18% 17% 14%
85th percentile or above
50th to 84th percentile
16th to 49th percentile
15th percentile or below
Caution: Data are combined across all districts and all survey types. Data do not represent any one district. Reliability for some scales was low.
8
Grade 10 Students’ Social and Emotional Competence: Baseline
Self-a
waren
ess
Self-m
anag
emen
t
Social
awar
enes
s
Relatio
nship
skil
ls
Resp
Decisi
on-m
aking
0%
20%
40%
60%
80%
100%
6%13%
8% 8% 9%
29%
40%
35%30% 26%
39%
31%
36% 48%45%
26%16%
21%15%
20%
85th percentile or above
50th to 84th percentile
16th to 49th percentile
15th percentile or below
Caution: Data are combined across all districts and all survey types. Data do not represent any one district. Reliability for some scales was low.
9
Findings about Student Social and Emotional Competence Across All Districts at Baseline
Results for seventh grade students• Females rate themselves more highly then
males for self-management, social awareness, and relationship skills
• For self-awareness and social awareness, American Indian and White students rated themselves the highest, followed by multi-racial, Native Hawaiian, Black, Hispanic, and Asian students, and finally Alaska Native students.
10
Findings about Student Social and Emotional Competence Across All Districts at Baseline
Results for tenth grade students• Females rate themselves more highly then
males for responsible decision making• American Indian students rated themselves
higher than all other racial groups (Native Hawaiian and Alaska Native students were not included because n=14 and n=5).
11
Sample Grade 7 Subgroup Results: Social Awareness
Total
Male
Female
Whit
e
Africa
n Am
erica
n
Hispan
icAsia
n
Amer
ican
India
n
Alaska
Nat
ive
Native
Haw
aiian
Mult
i-rac
ial0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
0.2150265038027190.2113821138211380.218823529411765
0.09169407894736850.1420798065296250.168535507985049
0.330453563714903
0.0892857142857142
0.636363636363637
0.3076923076923080.259567387687188
0.333026042867020.3771454381210480.287058823529412
0.315378289473685
0.4519347037484890.351342167855929
0.265658747300216
0.321428571428572
0.181818181818182
0.179487179487180.316139767054909
0.2756395482830140.258355916892502
0.293647058823529
0.343338815789474
0.271765417170496
0.3098878695208970.252699784017279
0.357142857142857
0.136363636363636
0.307692307692308
0.26955074875208
0.1763079050472460.1531165311653120.200470588235294
0.249588815789474
0.1342200725513910.1702344546381240.1511879049676030.232142857142857
0.0454545454545455
0.2051282051282050.154742096505824
85th percentile or above
50th to 84th percentile
16th to 49th percentile
15th percentile or below
Caution: Data are combined across all districts and all survey types. Data do not represent any one district. Reliability for some scales was low.
12
Sample Grade 10 Subgroup Results: Responsible Decision Making
Total
Male
Female
Whit
e
Africa
n Am
erica
n
Hispan
icAsia
n
Amer
ican
India
n
Mult
i-rac
ial0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
0.06099058534588620.06350550381033020.05863708399366090.07217847769028870.103833271306290.07301107754279960.06164383561643830.10383327130629
0.0616113744075829
0.2906262791649610.3048264182895850.277337559429477
0.3530183727034120.353181987346483
0.3444108761329310.287671232876712
0.3531819873464830.364928909952607
0.3888661481784690.3759525825571550.40095087163233
0.3293963254593180.3066617045031630.3625377643504530.452054794520548
0.3066617045031630.350710900473934
0.2595169873106840.255715495342930.2630744849445330.2454068241469820.2363230368440640.2200402819738170.1986301369863010.2363230368440640.222748815165877
85th percentile or above
50th to 84th percentile
16th to 49th percentile
15th percentile or below
Caution: Data are combined across all districts and all survey types. Data do not represent any one district. Reliability for some scales was low.
13
Strengths Across Districts
• Even in a climate of fiscal austerity, districts have still managed to support SEL.
• All districts have expressed a desire to go to scale with SEL
• Willingness to take on SEL standards seriously• Focus on professional development• Efforts at aligning foundational efforts and early and
intensive interventions• Insight into the relationship between SEL and deep
learning and the whole child • Insight into the relationship between SEL and
disparities in discipline, attendance, and learning
14
Challenges: Multiple Initiatives
• In every district there was concern expressed about: the rapid pace of program turnover being overburdened with too many initiatives at
once “innovation fatigue”
15
Challenges: Understanding
• SEL is sometimes: seen as “soft” by some educators regarded as a distraction confused with PBIS, character education,
student support, and other initiatives
16
Challenge: Implementation
• School autonomy and variation present challenges for supporting high-quality SEL implementation
• Linking elementary-middle-high school SEL has been a challenge
17
Challenge: Turnover
• Who is the champion for the CDI? How to ensure sustainability when there is turnover in key staff?
18
Challenge: SEL Metrics
• Strategic use of SEL metrics can support high-quality SEL implementation
• Some districts do well with data use, but don’t have SEL-related metrics available
• Some districts have SEL metrics available, but have not systematically supported use of those data
19
CDI Evaluation Activities 2012–13
• Site visits in spring Interviews with key district stakeholders SEL lead / team phone interviews Document review: focus on evidence of CDI
implementation
• Districtwide staff SEL survey Central office: relevant staff Schools: Principals, teachers, student support staff
• Teacher ratings of Grade 3 students’ SEL• Grade 7 & 10 student self-report of SEL skills• No school case studies
20
Next Steps
• AIR evaluation team will contact districts to develop/revise data collection plans for 2012–13 by the end of December
21
David Osher, Ph.D.
Phone: 202-403-5373
Email: [email protected]
1000 Thomas Jefferson St., NW
Washington, DC 20007
Website: www.air.org