2010 compass
TRANSCRIPT
Where youth learn skil ls to thrive in l i fe. They...
find their
SPARK
learn leadership
and tips for
HIGHSCHO
OL
SUCCESS
nurture a
GROWTH
MINDSET
hone their
GOAL
MANAGEM
ENT
SKILLS
“I will teach those who didn’t come to Compass, to be a
better leader and to show their spark more. And also tell them
about other things that they missed.”
2010
COMPASS 2010 – PROGRAM EVALUATION REPORT
Applied Survey Research 2 | P a g e
The Compass Program
2010 A pilot of Sequoia Union High School District in partnership with Thrive Foundation for Youth
Evaluation Report
Prepared by: Thrive Foundation for Youth and Applied Survey Research
THRIVE FOUNDATION FOR YOUTH
1010 El Camino Real, Ste 250 Menlo Park, CA 94025 Phone 650.485.2944
Contact: Nicole C. Taylor, President & CEO
www.thrivefoundation.org www.stepitup2thrive.org
Principal Researcher: Applied Survey Research
APPLIED SURVEY RESEARCH
991 W. Hedding St., Ste 102 San Jose, CA 95126
Phone: 408.247.8319
Contact: Lisa Colvig-Amir, MA, Director of Evaluation
Vanessa Haug, MS, Senior Research Analyst Patricia Reyes, MPH, Project Manager
www.appliedsurveyresearch.org
Applied Survey Research 3 | P a g e
Table of Contents
TABLE OF CONTENTS .............................................................................................................................. 3
1. INTRODUCTION ................................................................................................................................... 4
Program Description ................................................................................................................................................ 4
Study Description .................................................................................................................................................... 5 Evaluation Questions ................................................................................................................................................... 6 Methods ....................................................................................................................................................................... 6
2. DEMOGRAPHICS OF YOUTH ............................................................................................................ 8
Characteristics of Compass Students ........................................................................................................................ 8
3. STUDENTS’ EXPERIENCES IN THE COMPASS PROGRAM ........................................................ 9
Findings from the Student Survey ............................................................................................................................ 9 Did Compass students feel there was any value or benefit in participating in the Compass program? ...................... 9 Did Compass students feel that the Compass program provided a supportive environment? ................................. 10 Did Compass students feel that topics were meaningful? ......................................................................................... 10
4. STUDENTS’ CHANGES IN BELIEFS AND INTENTIONS ........................................................... 11 The Five Framing Concepts ........................................................................................................................................ 11
5. STUDENTS’ ACADEMIC PERFORMANCE IN 9TH GRADE ......................................................... 12 School Success Techniques ........................................................................................................................................ 12 Academic Outcomes .................................................................................................................................................. 12 Attendance Outcomes ............................................................................................................................................... 15
6. SUMMARY OF FINDINGS ................................................................................................................. 16
APPENDIX A: THEORY OF CHANGE ................................................................................................. 17
APPENDIX B: VIGNETTES ................................................................................................................... 18
ATTENDANCE OUTCOMES
Applied Survey Research 4 | P a g e
1. Introduction
Compass is a 24-‐day summer program offered by Sequoia Union High School District (SUHSD) in San Mateo County for entering 9th grade students who have been below grade level and are most at risk of dropping out of school. Held daily on four school campuses, each day of the program consists of five classes: Summer Reading, Math, High School Success, Leadership, and P.E. In 2009, SUHSD contacted Thrive Foundation For Youth (Thrive) to assist in improving the program. Thrive promotes a research-‐based theory of change about ways to redirect youth’s life trajectories in order to reach their full potential. The theory of change, called Step-‐It-‐Up-‐2-‐Thrive (www.stepitup2thrive.org) has four components that are based on unique bodies of research:
(Benson, 2008): Sparks are inner passions or interests that catalyze personal growth on a Sparks -‐ trajectory towards thriving. Youth who have spark(s) and have adult support in growing them are less likely to experience depression or engage in acts of violence against others, and are more likely to have a sense of purpose, receive higher grades, and have better school attendance rates.
(Dweck, 2006): Mindset is the belief about one’s intelligence, abilities, and personality. Mindset -‐ Those who believe their intelligence, abilities, and personality are innate and carved in stone have a “fixed mindset,” whereas those who believe that these qualities about themselves can be developed through effort and practice have a “growth mindset.” By fostering a growth mindset, youth are more resilient in the face of challenges and have a love of learning, thus leading to greater life success.
: Reflecting on one’s strengths and identifying risk factors with the help Reflect on Thriving Indicatorsof a supportive adult can lead to more thoughtful planning towards the pursuit of goals. Youth build their self-‐reflection skills in order to explore areas that they want to develop and identify factors that stand in the way.
(Lerner, 2006 ): GPS (i.e., goal management skills) is a set of skills that enable youth to: select GPS – goals, pursue strategies to attain goals, and navigate around obstacles. When youth build their goal management skills, they are more likely to thrive, increasing in positive youth development outcomes and decreasing in risk behaviors.
Program Description Program Description The Thrive team partnered with teachers and consultants to design Step-‐it-‐up-‐2-‐Thrive curriculum that integrated the above elements. To assist teachers in the delivery of their Compass course curriculum, Thrive partnered with several Bay Area youth development groups, such as Math and English Complex Instruction Coaches and Live in Peace (L.I.P). Brief descriptions of the revised academic classes are provided below.
COMPASS 2010 – PROGRAM EVALUATION REPORT
5
– This class trained students in key study skills which included Cornell note taking, High School Successbinder preparation, expository reading techniques, goal management skills, ways to seek help, test-‐taking, memorization tips, and practice in classroom public-‐speaking. These study skills were integrated into each week’s theme which highlighted a specific element of brain development or research on thriving in adolescence. During the first week of class, students were introduced to adolescent brain growth, specifically around Dr. Carol Dweck’s research from Stanford University on Growth Mindset. The second week of class focused on emotions and learning emotional regulation techniques. In the third week, youth applied knowledge about the brain to goal management practice, integrating recent Dr. Richard Lerner’s research from the Institute for Applied Research in Youth Development at Tufts University. The concluding weeks of class covered brain care with an emphasis on sleep, memory, drugs, and plans for addressing challenges to high school success.
– This class introduced youth to the Search Institute research about their personal Leadership Classspark. The purpose of the class was to help students identify their personal dreams and catalysts for growth, and tie these interests to high school opportunities. The curriculum was designed to develop leadership skills. Once a week, Live In Peace, an East Palo Alto organization that works to reduce youth violence, presented a class lesson that considered approaches to thriving in challenging environments. The themes included Dreams and Dream Thieves; Friendship or Fearship; Anger, Fear & Pain; and managing that "stress" balloon inside each of us. Leadership practice included a focus on styles, living by a moral compass, and conflict resolution.
– Teachers assisted students in their summer reading and writing skills. They were Reading Classasked to incorporate concepts of thriving (sparks, growth mindset, indicators of thriving, and goal management) in the discussions of book characters.
– The purpose of the class was to give all youth a sense that they could succeed in math. Math Class The program was designed to focus on new algebraic information and problem-‐solving skills that were not dependent on a youth’s prior competence with math facts. In preparation for Compass, teachers were taught a method for working with students in groups, which takes advantage of the diverse strengths of all students. This method, called Complex Instruction, was developed by Dr. Pope at Stanford University’s School of Education. In 2010, SUHSD pulled the lowest performing math students out of this class and they received an alternate form of instruction. Those students had two periods per day of this alternate math approach and skipped the reading class.
Study Description Study Description Once the Step-‐it-‐up-‐2-‐Thrive curriculum was finalized, Thrive reached out to Applied Survey Research (ASR), a non-‐profit social research firm, to conduct a two-‐year evaluation of the Compass program (2009 and 2010) to determine the extent to which students and teachers benefited from the intervention, and the ways in which Thrive’s theory of change may hold true. The evaluation in 2009 was exploratory since the Compass Re-‐Design was in its first year of inception, and considered to be a pilot-‐test of the effectiveness of a newly developed curriculum. Findings associated with the 2009 evaluation (i.e., those obtained from student and teacher surveys, as well as student and teacher focus groups) were therefore
COMPASS 2010 – PROGRAM EVALUATION REPORT
6
instrumental in determining the extent to which Compass’ curricular activities were beneficial to both students and teachers. Ultimately, a number of programmatic refinements were made in 2010, ranging from the content of the curriculum (e.g., expanding topical areas) to the delivery of the curriculum (e.g., applying a unified approach across all teachers and schools to teaching the program’s five framing concepts). This report focuses on year-‐two results (2010) ONLY.
Evaluation Questions
Once the program’s theory of change had been defined (see Appendix A), a comprehensive list of evaluation questions was identified by Thrive Foundation. These research questions served to tap into 1) students’ experiences while participating in the program, 2) teachers’ experiences with the enhanced program, and 3) students’ academic and behavioral outcomes associated with their participation in the program.
Students’ Experiences in the Compass Program
• Did Compass students feel there was any value or benefit in participating in the Compass program?
• Did Compass students feel that the Compass program provided a supportive environment?
Students Changes in Behavior and Intent to Behave
• Did students change their beliefs and intentions around Personal Spark?
• Did students change their beliefs and intentions around Growth Mindset?
• Did students change their beliefs and intentions around School Success Techniques?
• Did students change their beliefs and intentions around Reaching Goals & Dreams?
Student Outcomes Associated with the Compass Program (To be answered in 2011)
• In what ways did Compass students show evidence that they were being successful in school?
• Did Compass students have better academic (e.g., GPA and number of completed courses) and behavioral outcomes (e.g., disciplinary incidents) than their peers?
Methods
To answer the evaluation questions above, three research methods were used in 2010. These included a student pre/post survey, student focus groups, and examination of school performance and attendance data. Each of these methods is described in more detail below.
Pre/Post Survey was designed with the collaboration of Thrive to measure students’ beliefs, awareness, and motivations around the program’s five framing concepts (i.e., Personal Spark, Growth Mindset, Leadership, School Success Tips, Reaching Goals & Dreams). All Compass students were invited to complete the survey in their schools’ computer labs on the first day of
COMPASS 2010 – PROGRAM EVALUATION REPORT
7
Compass instruction, and again a day before the end of the program. To accommodate students with low literacy skills and to ensure consistency in administration and interpretation of items, the survey was read aloud to each Compass class by a staff member from ASR. Students listened to each question and marked their individual response on their own version of the on-‐line survey. In all, 394 students completed a pre-‐survey, and 372 completed a post-‐survey.
Focus Groups: Staff from ASR conducted focus groups with Compass students to explore their impressions of the program in general, as well as to conduct a more in-‐depth discussion around the program’s five framing concepts (i.e., personal spark, growth mindset, leadership, school success tips, reaching goals and dreams). A total of 40 students were randomly selected from four different physical education classes (one in each high school).
Academics: To assess the extent to which the Compass program influenced youth’s academic progress and behavioral referrals during their first year in high school, ASR obtained specific student-‐level data from Sequoia Union School District for two consecutive academic years: 2010-‐2011. These data included behavioral and academic information for both Compass and non-‐Compass participants. Data requested from the school district included demographic and background information, such as race, English learner status, GPA, 8th and 9th grade CST scores, attendance, and misconduct (n=384 students). To compare Compass students to their peers, ASR used two types of analyses:
• “Similarly-‐situated” comparison group: ASR created a statistically-‐matched comparison group that matched the cohort of Compass students in terms of the prevalence of key characteristics such as 8th grade CST performance levels in math and English, ethnicity, English Learner status, free/reduced lunch, and gender. Because the Compass population was not itself homogeneous in terms of Math and English proficiency, two cohorts were created: a ‘Math’ cohort of the Compass and non-‐Compass (labeled as Cohort M), and an ‘English’ cohort of the Compass and non-‐Compass (labeled as Cohort E)(See Figure 1).
• “At-‐risk” subset (Compass vs non-‐Compass): Since some Compass students were academically proficient or advanced in 8th grade, and was not reflective of the intended target population, ASR created a second data set with those students removed. The resulting subset is Compass students and non-‐Compass students at the three lowest CST levels of proficiency (Far Below Basic, Below Basic, and Basic).
COMPASS 2010 – PROGRAM EVALUATION REPORT
8
Figure 1: Matched Cohorts For Randomized Control Study
Cohort Based on 8th grade MATH scores (Cohort M)
Cohort Based on 8th grade ENGLISH scores (Cohort E)
Compass students
Non-‐
Compass students
Compass students
Non-‐Compass students
Advanced Advanced Advanced Advanced
Proficient Proficient Proficient Proficient
Basic Basic Basic Basic
Below Basic Below Basic Below Basic Below Basic
Far Below Basic
Far Below Basic
Far Below Basic
Far Below Basic
2. Demographics of Youth
Characteristics of Compass StudentsCharacteristics of Compass Students
Overall, there were 455 Compass students in the populations (see Figure 1). They had the following characteristics:
• The majority of students were Hispanic/Latino (78%). Very few were white (9%), Pacific Islander (4%), or African American (3%).
• There was a fairly equal representation of males (52%) and females (48%).
• Just over half (53%) were English Learners indicating that they had limitied English Proficiency. Twenty-‐five percent of the Compass youth were Reclassified Fluent English Proficient, which means they previously had limited proficiency but had become fluent. Twenty-‐one percent were raised in househoulds in which the primary language spoken was English (i.e., English
Figure 1 – Compass students’ characteristics
Ethnicity
N=455 (2010) Hispanic/Latino 78%
Pacific Islander (other) 4%
White 9%
African American/Black 3%
Gender
N=455 (2010) Male 52%
Female 48%
English Proficiency
N=409 (2010) English Learner 53%
RFEP 25%
English only 21%
IFEP 1%
COMPASS 2010 – PROGRAM EVALUATION REPORT
9
only). Only one percent were from households in which a language other than English was used (IFEP).
When examining Compass students’ Math and English subject area proficiency from the previous year (i.e., 8th grade), more than one-‐third (35%) did not have a Basic level of proficiency in English, and 43% did not have a Basic level of proficiency in Math (Figures 2 below). Several students were either proficient or had advanced skills in English (21%) and Math (20%). These students were included in the course as potential role models.
Figure 2 –Compass students proficiency levels in 8th grade (2010 cohort)
3. Students’ Experiences in the Compass Program
Findings from the Student SurveyFindings from the Student Survey
Did Compass students feel there was any value or benefit in participating in the Compass program?
As indicated in the post survey data, the majority of students valued their time spent in the Compass program. Specifically, they thought that the Compass program would support them in high school because it made them feel more enthusiastic about learning. They were particularly excited by the subjects: “How the brain works” – A Thrive-‐related concept -‐ and “math skills.” Almost all of them would recommend the program to a friend.
12%
23%
44%
19%
2%
11%
32% 37%
17%
3%
0%
10%
20%
30%
40%
50%
Far below basic Below basic Basic Proficient Advanced
8th grade English CST
8th grade Math CST
COMPASS 2010 – PROGRAM EVALUATION REPORT
10
Eighty-‐nine percent of students (n=343) reported that what they had learned at Compass was going to help them while in high school.
More than three-‐quarters of all students (77%, n=346) reported that the program had helped them feel more enthusiastic/excited to learn in high school.
When asked to list the “coolest” thing they learned while attending the program, students most commonly reported discussions on how the brain works (as it relates to growth mindset), as well as discussions about the impact of drugs on brain functioning. The second “coolest” thing they learned was math skills.
Ninety percent (n=339) said that they would recommend Compass to a friend.
Did Compass students feel that the Compass program provided a supportive environment?
In general, the majority of students felt that their Compass teachers not only “cared” about them, but also believed in their potential to succeed in high school. For instance, three-‐quarters of the students (75%, n=347) felt that their teachers “cared” about them. As for students’ perceptions around their teachers’ belief in their potential for success, almost all students (88%, n=347) felt that their teachers “believed” that they could succeed.
In addition to the value placed on teachers’ instructional methods, students also appreciated specific content areas discussed in their classes. For example, in 2009, students most commonly talked about the books, stories, and new vocabulary they learned in their Reading Class. As for their High School Success Class, they most valued the discussions on how the brain works, as it relates to growth mindset.
Did Compass students feel that Step-‐it-‐up-‐2-‐Thrive topics were meaningful?
Students were asked to rate the meaningfulness of all Compass topics on a four-‐point scale (ranging from 1 “not at all meaningful” to 4 “very meaningful”). While all eight topics resonated with students at some level, students felt particularly drawn to Respect comes from within (78%), Leadership skills (78%), Friendship and fearship (76%) and Standing up to peer pressure (73%) (see Figure 3).
“[I learned that] you need sleep because basically your brain needs rest and your brain is basically in charge of everything you do, like thinking,
motion, senses.”
“I will start on all assignments the day they are assigned and I plan on
paying attention and asking teachers for help when I don't understand
something.”
“The coolest thing I learned at Compass is how the brain works and
functions when you take drugs or alcohol.”
“[In my] English class I read a book call "Monster." It was interesting. I
also learned about drugs, fractions, and how to follow the steps to get to
my dreams.”
COMPASS 2010 – PROGRAM EVALUATION REPORT
11
Figure 3 – Student Percentage Who Found Step-‐it-‐up-‐2-‐Thrive Topics Meaningful
4. Students’ Changes In Beliefs and Intentions
The Five Framing Concepts
Pre/post surveys showed that students did not change their beliefs or intentions on items related to developing a personal Spark, identifying Spark Champions, or developing a Growth Mindset. However, during focus groups, students reported a different reality, stating that they were more likely to: reach out to their teachers after class if they need one-‐on-‐one help, have a growth-‐mindset, and try harder. These comments reflected specific lessons related to developing Spark Champions and Growth Mindset.
During the focus groups, students were also asked to talk about each of the program’s five framing concepts. The objective here was to determine the extent to which students understood these concepts (and how they intended to apply this new knowledge. Based on students’ feedback, the majority of them grasped the core concepts. Specifically, Compass students recognized the different aspects of a leader, and that everyone can be one; they understood the importance of staying positive (i.e., having a growth mindset); they distinguished friendships from “fearships”; and, finally, identified ways they could further nurture their sparks and leadership skills while participating in school activities (See Appendix B).
10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Destructive language
Moral compass and bystander behavior
Dealing with anger, fear and pain
Communication and public speaking
Standing up to peer pressure
Friendship and fearship
Leadership skills
Respect comes from within
Percentage of students
COMPASS 2010 – PROGRAM EVALUATION REPORT
12
5. Students’ Academic Performance in 9th Grade
School Success Techniques
Survey findings showed no signficant changes related to: study skills, their ability to look for leadership roles in high school, and understanding the impacts of marijuana on the brain. However, students’ knowledge of two school success techniques had significantly improved by the time they concluded the program (i.e, knowing how to take notes and seeing themselves as a leader) (see Figure 5).
Open-‐ended survey questions also illustrated that as a result of the program, students planned on getting enough sleep while in high school -‐ another risk factor covered in class -‐ (87%; n=349), on using Compass’ study practices (84%; n=347), and asking for help when faced with a problem at home or school (83%; n=346). Focus group findings also showed that students felt more prepared academically as a result of their participation in the program.
Figure 5–Student Pre/Post Mean Scores on Concepts Related to School Success Techniques
Note: All changes are statistically significant (p < 0.05).
Academic Outcomes
SIGNIFICANT FINDINGS
Grade Point Average (GPA)
Compass students in Cohort E had significantly higher GPAs at the end of 9th grade than their similarly-‐situated peers (2.24 as compared to 2.03, respectively p<.05). Most notable of which is the 0.28 point difference between “at-‐risk” Compass students and their peers.
2.96 2.84 3.07 3.01
1.00
2.00
3.00
4.00
Students know how to take notes in class Students see themselves as a leader
2010 Pre mean score 2010 Post mean score
COMPASS 2010 – PROGRAM EVALUATION REPORT
13
Figure 6 – Students’ average GPA by the end of 9th grade (Cohort E)
Credits Earned
Course performance (in this case the number of credits earned) is an important factor taken into consideration when determining students’ risk for dropping out of high school. Ultimately, for 9th graders to be on track to graduate, they should have earned a total of 50 credits by the end of their 9th grade year.
Compass students earned an average of 58 credits by the end of 9th grade, exceeding the 50-‐credit minimum (significantly higher than their peers, p<.05) and illustrating that they were “on track” for high school graduation. Provided below is the breakdown by cohorts M and E, and by all students vs. “at-‐risk” students. The widest gap was observed with students in cohort E, where the average number of credits was higher by about 11 points for Compass students.
Figure 7 – Credits earned by the end of 9th grade
Note: (*) denotes a statistically significant difference, p<.05. Data based on 384 overall students (Compass and non-Compass), and 237 “at-risk” students (Compass and non-Compass).
2.24 2.09 2.03 1.81
0.00
1.00
2.00
3.00
4.00
All students* "At-risk" students*
Compass students
Non-Compass students
58 56
58 57 54
52 48
46
30
40
50
60
All students* "At-risk" students* All students* "At-risk" students*
Compass students
Non-Compass students
Note: (*) denotes a statistically significant difference, p<.05.
COMPASS 2010 – PROGRAM EVALUATION REPORT
14
CST Proficiency Levels
Compass students were more likely than their peers to increase their proficiency in English. For instance, 30% of Compass students had increased their proficiency in English by at least one level, as compared to 19% of non-‐Compass students (a statistically significant difference, p<.05).
Figure 8 – Percentage of students who improved their English CST proficiencies in 9th grade
Note: (*) denotes a statistically significant difference, p<.05.
Statistically significant improvements were also observed with the “at-‐risk” group of Compass students, in that more Compass students experienced an increase in CST level (37%, as compared to 21% of their peers), and fewer Compass students experienced a decrease in CST level (13%, as compared to 36% of their peers).
Figure 9 – Percentage of “at-‐risk” students who improved their English CST proficiencies in 9th grade
Note: (*) denotes a statistically significant difference, p<.05.
30%
19%
51%
19%
39% 42%
10%
20%
30%
40%
50%
60%
Increased by 1 level* Decreased by 1 level* Remained the same
Compass students
Non-Compass students
37%
13%
49%
21%
36%
43%
10%
20%
30%
40%
50%
60%
Increased by 1 level* Decreased by 1 level* Remained the same
Compass students
Non-Compass students
COMPASS 2010 – PROGRAM EVALUATION REPORT
15
Attendance Outcomes
The district’s data file included two variables pertaining to students’ attendance: number of days “present,” and number of days “possible.” To calculate the number of days missed, the researchers subtracted each student’s number of days present from number of days possible. For example, if a student had 180 days possible and was present on 170 days, this student had missed a total of 10 days. Provided next are findings pertaining to students’ attendance while in 9th grade.
SIGNIFICANT FINDINGS
On average, Compass students missed 12 periods in 9th grade, as compared to 14 periods for their non-‐Compass peers. Significant differences were noted for Compass students in cohort E, as seen in the figure below.
Figure 10– Students’ average number of missed periods (Cohort E)
Note: (*) denotes a statistically significant difference, p<.05. The overall sample size of Compass and non-Compass students was 384. The sample size of the Compass and non-Compass “at-risk” subset was 237.
12 13
16 17
5
8
11
14
17
20
All students* "At-risk" students*
Compass students
Non-Compass students
COMPASS 2010 – PROGRAM EVALUATION REPORT
16
6. Summary Of Findings
Thrive Foundation supported a Compass re-‐design anchored to the Quaglia Institute research (www.qisa.org) that students drop out of high school when they do not feel a sense of self-‐worth, a sense of purpose, and are actively engaged in classroom learning linked to their life experiences and direction. The Compass curriculum strived to improve youth’s skills, attitudes, and perceptions about themselves so that they would enter high school with increased motivation and skills to reach their potential.
Surveys and focus groups demonstrated that Compass students had a very positive learning experience. Students felt that the program was of value, helping them to feel more enthusiastic about high school. They also felt that the program had provided a supportive environment, believed that teachers cared for them, and believed in them. During the Compass program, students also began to change some of their beliefs about themselves and reported that they might behave differently in high school. Specifically, they reported that they would ask their teachers for help, use Compass study practices, have a growth mindset, try harder, and get enough sleep (All topics covered in the curriculum.) The question was then asked: “Would this shift lead to reduced risk factors for graduation that can be measured by better attendance, improved GPA, and reduced delinquency than a similarly matched control population?” Even lacking evidence of consistent reinforcement during the following school year, data from the short Compass intervention suggests “yes,” the methodology adds value. The data also suggests a need for continued reinforcement of Compass learnings throughout the school year in order to sustain students’ early positive gains.
To answer the overarching research question of whether students' participation in Compass translates into improved school outcomes, the researchers analyzed three domains of school success: academic, attendance and behavioral. The findings revealed that the Compass program showed significant impact on students' academic performance and attendance, but no impact was noted on students' behaviors.
Key take-‐aways include the following:
GPA: Compass students had significantly higher GPAs by the end of 9th grade than did non-‐Compass students. The most “at-‐risk” students registered the greatest difference with their similarly-‐situated non-‐Compass peers.
CREDITS EARNED: The overall population of Compass students had earned significantly more credits as compared to non-‐Compass students by the end of 9th grade.
CST SCORES: With regard to students' CST proficiencies in 9th grade, significantly more Compass students moved up one CST level in 9th grade than their similarly-‐situated peers.
ATTENDANCE: Finally, Compass students had significantly better 9th grade attendance than non-‐Compass students.
COMPASS 2010 – PROGRAM EVALUATION REPORT
17
Appendix A: Theory of Change
Students also:
• Feel supported to achieve goals
• Develop a sense of belonging at school
At-‐risk students participate in a 24 day program with four classes each day:
• High school success
• Summer reading
• Math
• Leadership
Teachers receive :
• Kickoff training to Thrive concepts
• Time to design the summer curricula that infuses Thrive concepts into the four core classes
• On site coaching
• Follow up group work
Students increase
AWARENESS about:
• School competencies: “I know what it takes to be successful in school”
• Growth mindset
• Personal spark
• Reaching Goals & Dreams
• School success techniques
• Leadership & Social emotional skills
Students change
BELIEFs about:
• School competencies: “I have lots of strengths and I can apply them”
• Growth mindset
• Reaching Goals & Dreams
• School success techniques
• Leadership & Social emotional
skills
Students increase
INTENTIONs about:
• School competencies: “ I would like to follow my HS success plan”
• Growth mindset
• Reaching Goals & Dreams
• School success techniques
Students show EVIDENCE of change:
• School competencies: “I have achieved my HS Success goals””
• Growth mindset
• Reaching Goals & Dreams
• School success techniques
• Leadership & Social emotional skills
• Reduced risk of drop out at the end of 9th grade (no Fs, at least 50 units, better than 80% attendance)
Students also
• Develop a love of learning
Program Activities Immediate Outcomes Longer Term Outcomes
COMPASS 2010 – PROGRAM EVALUATION REPORT
18
Appendix B: Vignettes
Personal Spark
What does it mean to students?
“Something that gets you up in the morning and something that you love to do. Something you are good at.”
“Your inspiration – what you like to do.”
“Your sparks will be diminished if you don’t overcome your fearships.”
“Something you can always fall back onto.”
[A spark] is something that makes you shine.”
Personal Spark
How will students use that knowledge?
“Become a soccer player, and practice every day to get better and better.”
“Help my community.”
“It will help me select my classes.”
“I will attend school clubs.”
Growth Mindset
What does it mean to students?
“Try to stay positive and do your best, and never have a fixed mindset.”
“If you can’t do something you have to keep on trying until you get it right. If you can’t get it right, ask somebody. Even though you have rough times, you can get over them.”
“It means having an open mind, and being open to new things.”
Growth Mindset
How will they use that knowledge?
“Find positive ways to solve problems.”
“[I will] not say ‘no’ the first time someone gives me a new opportunity.”
“If you fail a test you know that you need to study harder. [It does not mean] that you are always going to fail or that you are not smart enough.”
“It’s better to face reality than to hide all the time. Most likely, I’m the kind of girl that will face every problem whether they help me, so I know what I did wrong, and so I can do better next time.”
Leadership
What does it mean to students?
“When you are a leader you have to take responsibilities.”
“At first I thought that a leader was someone loud and in charge, but it’s someone who really listens to what everyone else is saying.”
“A leader helps people.”
Leadership
How will they use that knowledge?
“I learned in leadership class not to let anyone take control of you. Don’t let drama get in the way of your goals.”
“I learned what it takes to be a leader and when to let your balloon air out before it could explode.”
“How to make better friends and lead people in the right direction.”
School Success Techniques
What does it mean to students?
“How to take Cornell notes and be able to keep organized.”
“Grades depend on behaviors. Pay attention and don’t play around.”
“[I learned] how to control my emotions.”
“Try to manage your time with your homework.”
School Success Techniques
How will they use that knowledge?
“How to ask questions, take notes, be open minded, have a growth mind, sleep well.”
“To ask for advice from others and to go and look for help when struggling on a subject.”