2014 – 2015 evaluation of the boys & girls clubs of metro ... · 2014 – 2015 evaluation of...

2014 – 2015 Evaluation of the Boys & Girls Clubs of Metro Denver

2014 – 2015 Evaluation of the Boys & Girls Clubs of Metro Denver

For more information, please contact:

Jean Denious, PhD

[email protected]

303/839-9422 Ext. 121

For General Inquiries/Questions

p. 303-839-9422

f. 303-839-9420

OMNI Institute

899 Logan Street, Suite 600

Denver, CO 80203

www.omni.org

mailto:[email protected]

Table of Contents Introduction ....................................................................................................................................................... 3

Data Sources ................................................................................................................................................. 4

BGCMD Club Attendance and Demographic Data .......................................................................... 4

NYOI Survey Data ................................................................................................................................... 4

DPS School Data ...................................................................................................................................... 4

SAYO Data ................................................................................................................................................ 4

Sexual Health Program Participation Data ......................................................................................... 5

Analytical Approach .................................................................................................................................... 5

Control Variables ..................................................................................................................................... 6

Statistical Significance ............................................................................................................................ 6

Results Section 1: Analysis of 2014-15 Outcomes for the Core Sample ............................................. 8

Description of the Core Sample ................................................................................................................ 8

Description of Analyses .............................................................................................................................. 9

Outcomes for the Core Sample by Club Attendance Frequency....................................................... 9

NYOI Survey Outcomes by Club Attendance .................................................................................. 10

SAYO Outcomes for the Core Sample by Club Attendance ......................................................... 11

DPS School Outcomes for the Core Sample by Club Attendance .............................................. 11

DPS School Outcomes for the Core Sample by Club Attendance and Grade Level .................... 14

Relationship between DPS School Outcomes and NYOI Survey Outcomes for the Core Sample

....................................................................................................................................................................... 16

Results Section 2: Analysis of 2014-15 Outcomes by Sexual Health Program Participation ........ 17

Description of Sample ............................................................................................................................... 17

Description of Analyses ............................................................................................................................ 18

Outcomes by Sexual Health Program Participation ........................................................................... 18

NYOI Survey Outcomes by Sexual Health Program Participation .............................................. 18

SAYO Survey Outcomes by Sexual Health Program Participation ............................................. 19

DPS School Outcomes by Sexual Health Program Participation ................................................. 20

Results Section 3: 2015 DPS School Outcomes for Club vs. Comparison Youth ............................. 22

Description of the Sample ........................................................................................................................ 22


Results Section 4: Change in Outcomes 2014 – 2015 .......................................................................... 24



Change in NYOI Survey Outcomes .................................................................................................... 24

Change in DPS School Outcomes ...................................................................................................... 25

Results Section 5: Change in Outcomes 2013-2015.............................................................................. 26


Prepared by OMNI Institute ii


Change in DPS School Outcomes ...................................................................................................... 26

Conclusions and Next Steps ........................................................................................................................ 28

Summary of Key Findings ........................................................................................................................ 28

Limitations ................................................................................................................................................... 28

Recommendations ..................................................................................................................................... 29

Appendix I: NYOI Survey Scales and Items ........................................................................................... XXX

Appendix II: SAYO Scales and Items ......................................................................................................... XLI

Program Experiences Scales ............................................................................................................... XLI

Sense of Competence Scales ............................................................................................................ XLII

Future Planning and Expectations Scales (Grades 6-12 only) .................................................. XLIV

Appendix III: Detailed Data Tables .......................................................................................................... XLV

Prepared by OMNI Institute iii

Figures Figure 1: Survey Outcomes for the Core Sample that Differ by 2015 Club Attendance Frequency

........................................................................................................................................................................... 11

Figure 2: Average School Attendance by Club Attendance Frequency ............................................. 12

Figure 3: Suspensions by Club Attendance Frequency .......................................................................... 13

Figure 4: Behavior Incidents by Club Attendance Frequency .............................................................. 13

Figure 5: PARCC Math Proficiency by Club Attendance Frequency .................................................. 14

Figure 6: NYOI Survey Outcomes that Differ by 2015 Sexual Health Participation ....................... 19

Figure 7: SAYO Survey Outcomes by 2015 Sexual Health Participation ........................................... 20

Figure 8: Average School Attendance by Sexual Health Program Participation ............................... 21

Figure 9: School Outcomes by Club Participation................................................................................... 23

Figure 10: Change in School Attendance Rate, 2014 to 2015 ............................................................. 25

Figure 11: Change in School Attendance Rate, 2012-13 to 2014-15 ................................................ 27

Prepared by OMNI Institute

1

Tables Table 1: Club Representation for the Core Sample .................................................................................. 8

Table 2: Club Member Gender by Grade Level for the Core Sample .................................................... 9

Table 3: Club Member Race/Ethnicity by Grade Level for the Core Sample ...................................... 9

Table 4: Summary of NYOI Survey Outcomes for the Core Sample by Club Attendance

Frequency ........................................................................................................................................................ 10

Table 5: Summary of SAYO Survey Results by Club Attendance Frequency .................................... 11

Table 6: Summary of DPS Survey Results by Club Attendance Frequency ....................................... 12

Table 7: Average School Attendance by Club Attendance Frequency and Grade Level ................ 14

Table 8: Proportion of Club Members Proficient on the PARCC English Test, by Club Attendance

Frequency and Grade Level ......................................................................................................................... 15

Table 9: Proportion of Club Members Proficient on the PARCC Math Test, by Club Attendance

Frequency and Grade Level ......................................................................................................................... 15

Table 10: Correlations between 2015 School and NYOI Survey Outcomes ..................................... 16

Table 11: Club Member Gender by Grade Level for Sexual Health Program Participants ............. 17

Table 12: Club Member Race/Ethnicity by Grade Level for Sexual Health Program Participants 17

Table 13: Club Representation for Sexual Health Program Participants ............................................ 18

Table 14: Summary of NYOI Survey Results by Sexual Health Participation .................................... 18

Table 15: Summary of SAYO Survey Results by Sexual Health Program Participation .................. 19

........................................................................................................................................................................... 20

Table 16: Summary of Survey Results by Sexual Health Program Participation .............................. 20

........................................................................................................................................................................... 21

Table 17: Summary of School Outcomes by Club Participation (Members vs. Nonmembers) ...... 22

Table 18: Change in NYOI Survey Outcomes, 2014 to 2015 ............................................................... 24

Table 19: Change in School Outcomes, 2014 to 2015 .......................................................................... 25

Table 20: Change in School Outcomes, 2012 to 2015 .......................................................................... 27

Table 21: 2015 Survey Outcomes by Club Attendance Frequency ................................................. XLV

Table 22: 2015 School Outcomes by Club Attendance Frequency ................................................. XLV

Table 23: 2015 Survey Outcomes by Sexual Health Participation ................................................. XLVI

Table 24: 2015 School Outcomes by Sexual Health Participation ................................................. XLVI

Table 25: Differences between Club Members and Nonmembers in 2015 School Outcomes XLVII

Table 26: Two-Year Change in Survey Outcomes, 2014 to 2015 ................................................. XLVII

Prepared by OMNI Institute 2

Table 27: Two-Year Change in School Outcomes, 2014 to 2015 .................................................. XLVII

Table 28: Three-Year Change in School Outcomes, 2012-13 to 2014-15 .................................. XLVIII

Table 29: Correlations between 2015 Survey Outcomes and 2014-15 School Outcomes ..... XLVIII


Introduction Since 2005, the Boys & Girls Clubs of Metro Denver (BGCMD) have partnered with OMNI

Institute (OMNI) to conduct longitudinal and cross-sectional evaluation efforts that are

systematic and examine overall Club impact on a broad range of desired outcomes. BGCMD

targets high-risk youth who are exposed to numerous risk factors, including substance use and

abuse, gang activity, delinquency, crime, and low graduation rates. To counter these risk factors,

the Clubs regularly offer programs related to academic achievement, leadership, self-esteem

building, and healthy living. The Clubs also strive to offer a safe and supporting monitored

environment as an alternative to youth being unsupervised at home or when out with friends.

In 2015, the Boys & Girls Clubs of America (BGCA) partnered with local Clubs to administer the

National Youth Outcomes Initiative (NYOI) survey to Club members across the United States.

BGCMD was funded to participate in the national evaluation, and therefore utilized the NYOI

survey for their local evaluation. BGCMD also partnered with Denver Public Schools (DPS) to

obtain academic outcome data for a subset of Club members and a group of comparison

students who did not attend the Clubs. The goal of the BGCMD evaluation is to answer these

primary questions:

1. Do youth who attend the Club frequently have better school and NYOI socio-

emotional outcomes than youth who attend less frequently?

2. Do Club members who participate in Sexual Health programming have better school

and NYOI socio-emotional outcomes than members who do not participate?

3. How do school outcomes for Club members compare to similar youth who did not

attend the Clubs?

To allow for a deeper understanding of outcomes for youth who attend the Club, this year’s

evaluation investigated the following additional questions:

1. Are there grade-level differences in school outcomes for Club members, and how do

they compare to grade-level differences for similar youth who did not attend the

Clubs?

2. How do outcomes change over time for Club members with consecutive years of data?

Outcomes of interest reflect the Boys & Girls Clubs’ goals of fostering academic success,

promoting good character and citizenship, and promoting healthy lifestyles. Several data sources

were utilized to answer the evaluation questions; each source is described below.


Data Sources

BGCMD Club Attendance and Demographic Data

Member data from the KidTrax and Cayen member tracking systems were provided by BGCMD

to OMNI. These data included demographic information for Club members as well as Club

members’ club attendance during the 2014-15 school year. Club attendance data was either

provided as the number of total visits/days or as a range of days attended, which included 60-80

days and 90 or more days. BGCMD goals for youth are consistent across their Club sites –

BGCMD aims to build stronger communities by helping youth to be academically successful and

career-ready, to make healthy choices, and to develop strong character and leadership skills.

NYOI Survey Data

BGCA provided OMNI with data collected from youth across the BGCMD sites. The survey

included several measures related to Boys & Girls Club programming outcomes, such as feelings

of support from staff at the Club, physical and emotional safety, healthy eating and exercise,

conflict resolution skills, commitment to school, and drug and alcohol use. Two versions of the

survey were administered: one for youth ages 9 to 12 and one for youth ages 13 and older. The

teen version of the survey included risk behavior questions related to sexual activity, items about

school and delinquency, and items pertaining to substance use that were not included on the

youth version. The items from the 2015 survey included in analyses can be found in Appendix I.

DPS School Data

Through partnership with DPS, BGCMD obtained school data for a subset of Club members

whose parents had given their permission for the data to be shared. The data included these

outcomes: 2014-15 average attendance rate (days attended school/total number of school days),

count of expulsions and suspensions in 2014-15, 2014-15 weighted grade point average (GPA),

2014-15 CMAS Partnership of Assessment of Readiness for College and Careers (PARCC)

English Language Arts and Mathematics Assessment Achievement Results. In addition to school

data obtained for Club members, de-identified student-level data were provided for students

who did not attend BGCMD (“comparison data”). Student comparison data were matched by

DPS to Club members’ data on key characteristics including race/ethnicity, grade level, and

outcomes from the previous academic year.

SAYO Data

DPS also provided OMNI with data from the Survey of Academic and Youth Outcomes (SAYO)

for BGCMD youth. The SAYO was developed by the National Institute Out-of-School Time

(NIOST). This was the first year data from SAYO was collected. DPS provided OMNI with pre-


test and post-test scores for eight SAYO scales. These scales and the items that comprise each

scale can be found in Appendix II. Scale scores range from 1-4, with 4 being the highest and most

desirable rating.

Sexual Health Program Participation Data

In addition to Club attendance data, BGCMD provided OMNI with Sexual Health program

participation rosters. These rosters indicated youth who participated in one or more Sexual

Health activities/programs. Sexual Health program participation was analyzed based on whether

or not Club members had participated in Sexual Health programming (Participation in Sexual

Health programming vs. No Participation in Sexual Health programming).

Analytical Approach Previous BGCMD evaluation efforts have determined that analyses based on length of Club

membership were not beneficial over and above analyses based on number of days spent at the

Club. That is, membership length alone has not been shown to distinguish youth outcomes, likely

due to the fact that participation levels vary greatly across youth regardless of how long they

have been members. It is also important to note that past evaluations have taken a broader

approach by including data from as many youth as possible across multiple school districts.

These approaches had several limitations, including large amounts of missing data, inability to

compare school outcomes across school districts due to dissimilar matching processes, and

inability to link school data to survey data.

Starting in 2014, the evaluation approach was consequently streamlined by focusing primarily on

a targeted group of youth, those with at least 60 days of attendance and DPS school data, in

order to enable a more sound examination of relationships of Club engagement and school and

socio-emotional outcomes. Specifically, Club members were included in the evaluation only if

they attended the Club 60 days or more and had DPS school data, with the exception of Section

2: Analysis of 2014-15 Outcomes by Sexual Health Program Participation. Section 2 does not

have the additional criteria of having DPS school data since the majority of youth who

participated in Sexual Health programming were in non-DPS BGCMD program sites1.

Some analyses also examined differences in outcomes among members who attended the Club

moderately frequently (60 – 89 days) or frequently (90+ days). The groups were created based

on cut off points of interest to BGCMD.

1 BGCMD sites that provided data for Sexual Health Program Participation included Denver Broncos, William E. Cope, J. Churchill Owen, George M. Wifley, Suncore (previously Commerce City), Jack A. Vickers, Robert M. Shopneck, and Jefferson County.


Using Club IDs and DPS Student IDs, data were merged across the data sources noted above as

well as with previous years of data in order to create a final file for analysis. Resulting sample

sizes for analyses are included at the beginning of each reporting section.

Analyses were conducted on level of Club participation, as measured by the number of days

members attended the Club during the 2014-15 school year.

Club Attendance and Demographic Data: BGCMD provided Club attendance and demographic

data for youth who attended the Club 60 days or more (N = 1744).

DPS School Data: DPS school data were matched to BGCMD Club attendance and demographic

data which resulted in 591 matched records. This group of youth is considered the “core sample”

and comprises the primary group of youth included in the evaluation.

NYOI Survey Data: The core sample (i.e., youth with Club attendance and school data) was

merged with 1,528 NYOI survey respondents which resulted in 447 matched records. Analyses

focusing on NYOI outcomes included the 447 youth from this sample.

SAYO Data: Analyses focusing on SAYO outcomes in section 1 included participants in the “core

sample” who also completed the SAYO survey consisting of 276 matched records. Analyses

examining SAYO outcomes in Section 2 included participants who reported 60 or more days of

Club attendance, participated in the Sexual Health programming, and completed the SAYO

survey (431 matched records).

Sexual Health Program Data: Sexual Health Program rosters were matched to Club Membership

and Attendance, which resulted in 211 matched records.

Control Variables

Some variables were related to most of the survey and school outcomes. To increase the

likelihood that any outcome differences observed were due to the variables of interest and not

other variables, it was important to control statistically for other variables’ influence. The grade

of the Club members was statistically controlled for in analyses of the 2015 survey data and the

school data, except for analyses that were specifically assessing grade level differences. These

analytic precautions were taken because preliminary analyses indicated that youth differed on

outcomes based on their grade regardless of their attendance at the Club.

Statistical Significance

Statistical tests, such as ANOVA, linear regression and chi-square, are tests of statistical

significance. Statistical significance is a way of representing the probability that differences

between observed group means are real differences. Tests of statistical significance calculate the

probability (p-value) that a difference in scores is due to chance.


P-values range from 0.0 to 1.0 with smaller values indicating greater levels of significance. That

is, the p-value represents the percent likelihood that an observed change or difference in scores

is due to chance. The standard benchmark in social science is a p-value of less than 0.05,

indicating less than 5% likelihood that an observed change or difference in scores is due to

chance.

Symbols are used throughout this report to indicate the level of significance wherever results

were statistically significant or approached statistical significance. The symbols represent p-

values as follows:

^ = .05 < p < .10

* = .01 < p < .05

** = p < .01

Tests of statistical significance were conducted to examine whether the pattern of change on an

outcome differed significantly across two or more groups (e.g., is the mean for the moderately

frequent attendance group different from the mean of the frequent attendance group?).

Throughout this report, the p-value symbols are used in the data tables to denote statistically

significant results.


Results Section 1: Analysis of 2014-15 Outcomes for the Core Sample

Description of the Core Sample More than 10,000 youth attend the Boys & Girls Clubs of Metro Denver annually. As described

above, the majority of this year’s evaluation focused on Club members who attended a Club for

60 days or more in 2014-15 and were in the 2014-15 DPS school dataset. These youth are

considered to be the ‘core sample’ of youth for 2014-15 analyses (N = 591).

Club members from 11 sites were included in the ‘core sample’ evaluation. Club members who

attend the Boettcher, Suncor (previously named Commerce City), Jefferson County, and

Shopneck sites do not attend DPS and therefore do not have DPS Academic data and were not

included in the ‘core sample’ analyses.

Table 1: Club Representation for the Core Sample

N % of Sample

Denver 71 12.0%

Cope 63 10.7%

Johnson 59 10.0%

Owen 83 14.0%

Wilfley 51 8.6%

Force - Beacon 45 7.6%

Cole - Beacon 53 9.0%

Place - Beacon 45 7.6%

Johnson - Beacon 42 7.1%

Noel-Beacon 41 6.9%

Vickers 38 6.4%

Total 591 100%

The demographic characteristics of the ‘core sample’ of Club members included in this section of

the evaluation are presented below. Demographic information includes gender and

race/ethnicity. Club members’ gender and race/ethnicity are reported in Tables 2 and 3 below.

Slightly more males than females were included in the evaluation and the predominant

ethnic/racial group in the sample was Hispanic/Latino. Age was available for only 59.1% of the

core sample. The average age for the 349 youth with Age information was 12.7 years.


Table 2: Club Member Gender by Grade Level for the Core Sample

Elem. (3 - 5)

MS (6 - 8)

HS (9 - 12)

Total

Female Count 111 87 70 268

% within Grade level 43.4% 45.8% 49.0% 45.5%

Male Count 145 103 73 321


All Count 256 190 143 589


Table 3: Club Member Race/Ethnicity by Grade Level for the Core Sample

Elem. (3 - 5)

MS (6 - 8)

HS (9 - 12)

Total

Hispanic or Latino Count 165 85 72 322


African-American/Black Count 50 71 49 170


White Count 12 10 6 28


Multi-Racial or Other Count 12 12 11 35 % within Grade level 4.7% 6.3% 7.7% 5.9% Asian Count 5 6 2 13


Am. Indian or Alaska Native Count 5 5 3 13


All Count 256 190 143 589


Description of Analyses Analyses of covariance (ANCOVAs) were conducted to analyze 2015 survey and mean-level

school outcomes by Club attendance frequency group controlling for grade. Chi-square analyses

were conducted to determine if the percentage of students who had proficient PARCC tests

differed by Club attendance or grade level. Lastly, correlation analyses were conducted to

analyze associations between school and survey outcomes.

Outcomes for the Core Sample by Club Attendance Frequency Club members were split into two groups based on their attendance at the Club during the

2014-15 school year. Moderately frequent attenders were at the Club between 60 and 89 days;

frequent attenders were defined as those who attended the Club at least 90 days during this


time period 2. It is an important caveat that all of the analyses in this section are correlational in

nature. That is, statistically significant results indicate that school or survey outcomes are related

to Club attendance frequency but cannot be interpreted as Club attendance having caused the

outcomes.

NYOI Survey Outcomes by Club Attendance

General Linear Models (GLMs) analyses were conducted to test if outcomes on the NYOI survey

differed based on the two levels of Club attendance. All of these analyses controlled for grade.

There was one finding that trended towards significance, as delineated in Table 4 below. This

finding is further illustrated in Figure 1.

Table 4: Summary of NYOI Survey Outcomes for the Core Sample by Club Attendance Frequency

Difference by Club Attendance Frequency No Difference by Club Attendance Frequency

Leadership skills Club support

Club belonging

Emotional safety at the Club

Physical safety at the Club

Number of Club areas avoided

Fun at the Club

Validation/recognition

Expectations from adults

Conflict resolution skills

Community service at the Club (Youth)

Community service at the Club (Teen)

Community service at the school (Teen)

Importance of school

Healthy eating

Unhealthy eating

As shown in Figure 1, frequent attenders reported having stronger leadership skills than

moderately frequent attenders, on average, p = .08. Note that leadership skills were measured on

a 4-point scale, with lower scores indicating stronger leadership skills; both attendance groups

reported strong leadership skills (on average, means are near 1.6 out of 4).

2 The evaluation sample was limited by BGCMD to include only youth who attended the Club at least 60 days in 2014-15.


Figure 1: Survey Outcomes for the Core Sample that Differ by 2015 Club Attendance Frequency

SAYO Outcomes for the Core Sample by Club Attendance

GLM analyses were conducted to evaluate whether SAYO outcomes differed by Club

attendance frequency controlling for grade. As shown in Table 5, no SAYO outcomes varied as a

function of Club attendance.

Table 5: Summary of SAYO Survey Results by Club Attendance Frequency


Challenge

Supportive adults

Choice/autonomy

Sense of competency as a learner

Sense of competency as a reader

Sense of social competency

Future planning

Future expectations

Youth believed program helped academic skills

Youth believed program helped social skills

DPS School Outcomes for the Core Sample by Club Attendance

Similar analyses were conducted to evaluate whether school academic outcomes differed by

Club attendance frequency. As shown in Table 6, four outcomes varied as a function of Club

attendance: school attendance, suspensions, behavior incidents, and math proficiency. All of

these findings favored frequent attenders; the results are displayed in Figures 2-5 to illustrate

the pattern of results.

1.681.57

1.0

2.0

3.0

4.0

Leadership skills

Moderately frequent attenders Frequent attenders


Table 6: Summary of DPS Survey Results by Club Attendance Frequency


Average school attendance GPA

Suspensions English proficiency

Behavior incidents

Math proficiency

As shown in Figure 2, BGCMD members who came to the Club at least 90 days had higher

average school attendance than members who came to the Club between 60 and 89 days, p <

.05. As a reminder, it would be inappropriate to conclude that attendance in either school or the

Club caused more frequent attendance at the other location. These results could simply indicate

that some youth are more likely to show up both at school and at the Club.

Figure 2: Average School Attendance by Club Attendance Frequency

As shown in Figure 3, BGCMD members who came to the Club at least 90 days had fewer

suspensions than members who came to the Club between 60 and 89 days, p < .05.

0.92 0.93

0.5

0.6

0.7

0.8

0.9

1.0

School attendance



Figure 3: Suspensions by Club Attendance Frequency

As shown in Figure 4, BGCMD members who came to the Club at least 90 days also had fewer

behavior incidents than members who came to the Club between 60 and 89 days, p = .09.

Figure 4: Behavior Incidents by Club Attendance Frequency

As shown in Figure 5, BGCMD members who came to the Club at least 90 days had higher

PARCC math proficiency scores than members who came to the Club between 60 and 89 days,

p = .06.

2.11

1.39

0.0

1.0

2.0

3.0

Suspensions


0.14

0.07

0.0

0.1

0.2

0.3

0.4

0.5

Behavior incidents



Figure 5: PARCC Math Proficiency by Club Attendance Frequency

DPS School Outcomes for the Core Sample by Club Attendance and Grade Level A series of analyses were conducted to evaluate if there were significant interactions between

Club attendance frequency and grade level on school outcomes. A significant interaction would

indicate that the impact of Club attendance frequency differs for BGCMD members based on

whether they are in elementary, middle, or high school.

Although average school attendance differed by Club attendance (higher average school

attendance among frequent Club attenders, p = .03) and by grade level (lower average school

attendance for higher grade levels, p < .01), the interaction between the two was not statistically

significant. In other words, the pattern by grade level did not differ between the two Club

attendance groups; see Table 7.

Table 7: Average School Attendance by Club Attendance Frequency and Grade Level

Club Attendance Frequency

Moderately Frequent Frequent

Grade Level

Elem. 0.94 0.95

MS 0.92 0.93

HS 0.90 0.91

GPA and behavior incidents did not differ significantly by Club attendance, grade level, or the

interaction between Club attendance and grade level. Although suspensions differed by Club

2.16 2.18

0.0

1.0

2.0

3.0

Math proficiency



attendance (greater number of suspensions among moderately frequent Club attenders, p = .04),

they did not differ by grade level, and the interaction between Club attendance and grade level

was not significant.

For each PARCC subject test (English and math), Club youth were put into a “not proficient”

group if they scored unsatisfactory or partially proficient, or into a “proficient” group if they

scored proficient or advanced. Then analyses were conducted to evaluate if the proportion of

members with proficient scores differed by grade level, Club attendance level, and by the

interaction of those two factors. English proficiency differed significantly by grade level, but did

not differ by Club attendance or the interaction between Club attendance and grade level.

Regardless of Club attendance frequency, members in higher grade levels had a lower proportion

of proficient scores on the English test than elementary students (p < .01; see Table 8 below).

Table 8: Proportion of Club Members Proficient on the PARCC English Test, by Club Attendance Frequency and Grade Level


Moderately

Frequent

Frequent Total

Grade Level

Elem. 20% 19% 19%

MS 15% 5% 8%

HS 27% 16% 18%

Total 19% 14% 15%

There was a marginally statistically significant effect of grade level for the PARCC math test, p =

.052. Regardless of Club attendance frequency, members in higher grade levels had a lower

proportion of proficient scores on the math test than elementary students; see Table 9 below.

Table 9: Proportion of Club Members Proficient on the PARCC Math Test, by Club Attendance Frequency and Grade Level


Moderately

Frequent

Frequent Total

Grade Level

Elem. 6% 14% 12%

MS 15% 10% 11%

HS 15% 7% 9%

Total 11% 11% 11%


Relationship between DPS School Outcomes and NYOI Survey Outcomes for the Core Sample Correlation analyses were conducted to test for significant relationships between the 2014-15

school outcomes and 2015 NYOI outcomes. Although there were a few scattered correlations

between several of the outcomes (see Table 29 in Appendix II for the full set of correlation

results), there was a clear pattern for two of the survey outcomes. As shown in Table 10, one

survey outcome (conflict resolution skills) was positively related to school outcomes. BGCMD

members who had more conflict resolution skills tended to perform better in school. Conversely,

youth who engaged in unhealthy eating (soda and fried foods) tended to not perform as well in

school.

In Table 10 a plus sign (+) indicates that higher values on one outcome are significantly related to

higher values on the other outcome. A negative sign (-) indicates that higher values on one

outcome are significantly related to lower values on the other outcome.

Table 10: Correlations between 2015 School and NYOI Survey Outcomes

Conflict Resolution Skills Unhealthy Eating

Attendance + —

GPA + —

Math Proficiency + —

English Proficiency + —

Note: + = positive correlation, - = negative correlation.

For these correlational analyses, it is not possible to tell whether there is a causal relationship.

In other words, these results cannot indicate whether doing better in school led to more conflict

resolution skills, or whether youth with strong skills in turn perform better in school. However,

these relationships are still important because they suggest areas for BGCMD to target with

programming (e.g., teach healthy eating and conflict resolution skills) and then measure over time

to look for a causal relationship.


Results Section 2: Analysis of 2014-15 Outcomes by Sexual Health Program Participation

Description of Sample The demographic characteristics of the Club members who had at least 60 days of Club

attendance and participated in Sexual Health programming are presented below. Demographic

information includes gender, race/ethnicity, and Club representation. These characteristics are

reported in Tables 11-13 below. Slightly more females than males were included in the

evaluation and the predominant ethnic/racial group in the sample was Hispanic/Latino. The

majority of youth (78%) who participated in a Sexual Health program had attended the Club at

least 90 days during the 2014-15 school year.

Table 11: Club Member Gender by Grade Level for Sexual Health Program Participants

Elem. (3 - 5)

MS (6 - 8)

HS (9 - 12)

Total

Males Count 18 29 46 93


Females Count 17 67 33 117


All Count 35 96 79 210


Table 12: Club Member Race/Ethnicity by Grade Level for Sexual Health Program Participants

Elem. (3 - 5)

MS (6 - 8)

HS (9 - 12)

Total

Hispanic or Latino Count 16 53 46 115


African-American/Black Count 14 26 18 58


White Count 2 10 6 18


Multi-Racial or Other Count 3 6 6 15


Asian Count 0 1 2 3


American Indian or Alaska Native

Count 0 0 1 1


All Count 35 96 79 210



Table 13: Club Representation for Sexual Health Program Participants

60-89 days Count 47

% 22.3%

90 or more days Count 164

% 77.7%

All Count 211

% 100.0%

Description of Analyses ANCOVAs were conducted to compare survey outcomes for Club members who participated in

Sexual Health programming versus those who did not participate in Sexual Health programming,

controlling for grade of the students.

Outcomes by Sexual Health Program Participation

NYOI Survey Outcomes by Sexual Health Program Participation

Some of the NYOI survey outcomes differed by Sexual Health program participation, but most

did not; see Table 14.

Table 14: Summary of NYOI Survey Results by Sexual Health Participation

Difference by Sexual Health Participation No Difference by Sexual Health Participation

Club belonging Club support

Emotional safety at the Club Physical safety at the Club

Healthy eating Fun at the Club

Volunteering at the Club (youth) Validation/recognition



Leadership skills

Unhealthy eating

Community service at the school (teen)

Volunteering at the Club (teen)

Importance of school

Number of Club areas avoided

The results that reached at least marginal statistical significance are presented in Figure 6.

Sexual Health youth participants reported that they volunteered more frequently at the Club

than non-Sexual Health youth participants, p = .009. The other findings favored non-Sexual

Health participants, such that:

On average, non-Sexual Health participants reported that they ate healthy foods more

frequently than Sexual Health participants, p = .07.


Non-Sexual Health participants reported feeling a greater sense of Club belonging than

Sexual Health participants, p = .02.

Non-Sexual Health participants reported feeling more emotionally safe at the Club than

Sexual Health participants, p < .001.

Figure 6: NYOI Survey Outcomes that Differ by 2015 Sexual Health Participation

SAYO Survey Outcomes by Sexual Health Program Participation

As shown in Table 15, only one SAYO outcome varied as a function of Sexual Health

participation.

Table 15: Summary of SAYO Survey Results by Sexual Health Program Participation


Choice/autonomy Challenge

Supportive adults

Sense of competency as a learner

Sense of competency as a reader

Sense of social competency

Future planning

Future expectations

Youth believed program helped academic skills

Youth believed program helped social skills

As shown in Figure 7, Sexual Health participants reported feeling greater choice/autonomy than

non-Sexual Health participants, p = .04.

3.013.23 3.32

2.79

3.50

3.14 3.12

2.52

1.0

2.0

3.0

4.0

5.0

Club volunteering (Y) Club belonging Emotional safety at theClub

Healthy eating

No Sexual Health participation Sexual Health participation


Figure 7: SAYO Survey Outcomes by 2015 Sexual Health Participation

DPS School Outcomes by Sexual Health Program Participation

Analyses were conducted to compare school outcomes for Club members who had participated

in Sexual Health programs versus members who had not participated. These results are

summarized in Table 16.

Table 16: Summary of Survey Results by Sexual Health Program Participation


School attendance GPA

Behavior incidents

Suspensions

Math proficiency

English proficiency

The significant finding was in favor of those who did not participate in the Sexual Health

program. As shown in Figure 8 below, non-Sexual Health participants had higher average school

attendance in 2015 than Sexual Health participants, p = .004, although both groups had high

attendance rates (94% and 91%, respectively).

3.17

3.47

1.0

2.0

3.0

4.0

Choice/autonomy



Figure 8: Average School Attendance by Sexual Health Program Participation

0.940.91

0.0

0.2

0.4

0.6

0.8

1.0

Average school attendance



Results Section 3: 2015 DPS School Outcomes for Club vs. Comparison Youth

Description of the Sample Section 3 of the report focuses on the relationship between participation at the Club (Club

members vs. nonmembers) and school outcomes. Denver Public Schools (DPS) provided

comparison data for this year’s evaluation. Club members were matched to nonmembers on the

following characteristics: free or reduced lunch eligibility, ethnicity, school, English Language

Learner status, grade, prior year’s TCAP reading proficiency, and the previous year’s misconduct

count (N = 1325). DPS school outcomes for these youth are compared to the ‘core sample’ of

youth for 2014-15 analyses (N = 591). It should be noted that the comparison group youths’

grade levels were not included in the data provided to OMNI, thus grade could not be controlled

for in these analyses.

Description of Analyses Several t-tests were conducted to assess the relationships between Club participation and school

attendance, misconduct and GPA. Club members’ 2015 school outcomes were compared to

outcomes for similar nonmembers.

There were three statistically significant differences in school outcomes between Club members

and nonmembers (see Table 17 below).

Table 17: Summary of School Outcomes by Club Participation (Members vs. Nonmembers)

Difference by Club Participation No Difference by Club Participation

School attendance Behavior incidents

English proficiency Suspensions

Math proficiency GPA

As shown in Figure 9 below, Club members had significantly higher levels of school attendance

than nonmembers, p < .001. The other two findings were in favor of nonmembers. Nonmembers

had significantly higher PARCC math proficiency scores (p = .048) and marginally significantly

higher PARCC English proficiency scores (p = .08).


Figure 9: School Outcomes by Club Participation

0.90

2.40 2.28

0.93

2.30 2.18

0.0

1.0

2.0

3.0

4.0

5.0

Average school attendance English proficiency Math proficiency

No Club participation Club participation


Results Section 4: Change in Outcomes 2014 – 2015

Description of the Sample Section 4 of the report focuses on analyzing change in outcomes between 2013-14 and 2014-

15. The ‘core sample’ of youth included in 2014-15 analyses were matched to the ‘core sample’

of youth included in 2013-14 analyses (i.e., Club members with at least 60 days of attendance in

2013-14 and 2014-15 and who had DPS data available in 2013-14 and 2014-15 (N=135). Of

those 135 Club members, 66 took the NYOI survey both years. Although these results are not

generalizable to all Club members, they may be reasonably representative of youth who attend

the Clubs frequently each year and attend DPS.

Description of Analyses Because the SAYO was not administered in previous years and PARCC and TCAP scores are not

comparable, change in outcomes for these measurements could not be assessed.

Paired-sample t-tests were conducted to evaluate how NYOI survey and DPS school outcomes

changed over time for Club youth who had data in 2014 and 2015. The survey analyses were

conducted for all Club members with available longitudinal data and who attended BGCMD at

least 60 or more days. A summary of the NYOI survey outcome changes is in Table 18; there

were no changes from 2014 to 2015.

Change in NYOI Survey Outcomes

Table 18: Change in NYOI Survey Outcomes, 2014 to 2015

Changed over Two Years Did Not Change over Two Years

Club support

Club areas avoided

Club belonging

Emotional safety at the Club

Physical safety at the Club

Fun at the Club

Validation/respect



Healthy eating

Unhealthy eating

Teen volunteering in school/community

Teen volunteering at the Club

Youth volunteering at the Club

Leadership skills


Change in DPS School Outcomes

Changes in school outcomes were analyzed between 2014 and 2015. A list of these outcomes is

included in Table 19. The sample was too small to test for change in expulsions (Number of

expulsions in 2014 = 1; number of expulsions in 2015 = 0).

Table 19: Change in School Outcomes, 2014 to 2015

Changed over Two Years Did Not Change over Two Years

School attendance Suspensions

GPA

The school attendance rate significantly declined for the entire sample (93.7% in 2014 to 91.3%

in 2015, p < .001); see Figure 10. Despite these declines, it is important to note that while there

was a statistically significant difference, the attendance rates were very good overall in both

years and not necessarily meaningfully different.

Figure 10: Change in School Attendance Rate, 2014 to 2015

93.7% 91.3%

0%

20%

40%

60%

80%

100%

School attendance

2014 2015


Results Section 5: Change in Outcomes 2013-2015

Description of the Sample Section 6 of the report focuses on analyzing change in outcomes for 2012-13, 2013-14, and

2014-15. Analyses were conducted for Club members with at least 60 days of attendance in

2012-13, 2013-14, and 2014-15 and who had DPS data available in 2012-13, 2013-14, and

2014-15 (N=135).

The N of students with data of the NYOI survey for all three years was too small to be able to

calculate change in the NYOI survey outcomes from 2013-2015 (N=9).

It is very important to note that only 25 Club members included in the evaluation had DPS

School Outcomes in 2013, 2014, and 2015. Therefore, care must be taken in drawing any

conclusions about BGCMD youth based on the results in this section. These results may be

generalizable to youth who attend the Club very frequently year after year, but likely do not

represent most BGCMD youth.

Description of Analyses A General Linear Models (GLM) was conducted to look at change in school attendance over

three years of data. The sample size was too small to conduct analyses of changes for the other

school outcomes (i.e., expulsions, suspensions, behavioral incidents and GPA).

Change in DPS School Outcomes

School attendance significantly decreased over time (see Table 20; Figure 11). Post hoc Tukey

tests indicated attendance rates for 2014-15 were significantly lower than 2012-13 and 2013-

14. As noted in the above section, this pattern was also observed for the larger sample of youth

with two years of data. Attendance rates did not differ between 2012-13 and 2013-2014.


Table 20: Change in School Outcomes, 2012 to 2015

Changed over Three Years Did Not Change over Three Years School attendance

Figure 11: Change in School Attendance Rate, 2012-13 to 2014-15

94.3% 94.0%89.8%

0%

20%

40%

60%

80%

100%

2012-13 2013-14 2014-15


Conclusions and Next Steps

Summary of Key Findings Findings from BGCMD’s 2014-15 evaluation were mixed, but indicate several positive outcomes

that are associated with more frequent attendance at the Clubs. Notable outcomes include

higher rates of school attendance, fewer behavior incidents and suspensions for Club members

who attended the Clubs 90 days or more as compared to Club members who attended the Club

for 60 – 89 days in 2014-15. Club members who attended the Club 90 days or more also

reported stronger leadership skills and higher scores on math proficiency tests.

Longitudinal analyses that examined NYOI outcomes among moderate and frequent Club

attendees across two years found no changes. This indicates that these outcomes remained

stable across time with no significant increases or decreases observed. An examination of school

attendance of Club members across three years found a significant decrease in attendance rates

in 2014-2015 compared to the previous two years. However, a relatively high attendance rate

was still observed.

Finally, analyses that examined differences between members who participated in a sexual

health program to those who did not indicated members who participated in the program at

BGCMD reported greater choice/autonomy and more frequent volunteering at the Club.

However, findings for participants were somewhat mixed considering they reported feeling less

emotionally safe at the Club, and a lower sense of belonging at the Club compared to those who

did not participate in the sexual health program.

Limitations There are several limitations to BGCMD’s current evaluation design, which leads to limited ability

to examine relationships across data sources (for example, relationships between school and

survey outcomes) and across years. This year’s evaluation aimed to focus on a group of Club

members who attend the Clubs most frequently and who had data available from multiple

sources. Despite efforts to streamline the evaluation approach, sample size across data sources

and across years remained too small to examine many potentially interesting and informative

outcomes. BGCMD should work to focus their evaluation approach so that relationships

between outcomes and over time can be analyzed for a more representative sample of Club

youth.

Additionally, BGCMD obtained data from a comparison sample of youth from Denver Public

Schools who were similar to BGCMD youth on several demographic characteristics but who did

not attend the Clubs. Comparison data for this year’s evaluation did not allow for a one-to-one

match between comparison students and Club members over multiple years, limiting the types of


comparisons that could be made between the BGCMD and nonmember group. In addition, a

significant portion of the main outcome variables changed across years (e.g. TCAP scores were

replaced with PARCC scores, NYOI survey items changed), which restricted the longitudinal

analyses that could be conducted.

Recommendations As in years past, OMNI continues to recommend a more systematic, controlled examination of

Club members. Specifically, rather than deriving the analytic sample(s) from those with available

data (a non-randomized sample), we strongly recommend that BGCMD implement a study design

whereby a randomly selected or representative subset of Club members is identified, then

recruited by BGCMD staff to participate in the evaluation. A sufficiently large number of

members should be targeted to allow for attrition and refusals, and efforts should be focused on

ensuring data is collected and obtained for these particular youth across sources. This approach

helps minimize the bias inherent in a sample that can only include those youth who participated

in the Club frequently enough to be present for survey administrations (especially across years).

That is, it better ensures that the sample is representative of the population of youth who attend

the Clubs with varying frequency.

It is very possible that with a randomized sample, more positive, significant and interpretable

relationships of Club engagement to outcomes would be observed, especially given the wider

variance in Club attendance that could be captured in a random sample. A greater number of

longitudinal analyses would also be possible to conduct, with sample sizes that can allow for

more confidence in changes observed over time for Club members.

Finally, this approach would better ensure that investment of resources and time yields useful

information for BGCMD. While the effort to track a group of members over time can be

intensive, other efficiencies may be realized, with net time and costs roughly equivalent to the

present approach. For example, this would allow BGCMD to focus its internal data collection

efforts on the randomized (and much smaller) sample versus all Club members, and allow for

more advanced planning with DPS and others responsible for pulling member and comparison

data. And, significantly fewer resources and time would be needed for the evaluator to clean,

merge, and prepare data for analysis.


XXX

Appendix I: NYOI Survey Scales and Items Support at the Club Scale3

About how many staff at the Boys & Girls Club...

1. Pay attention to what’s going on in your life? (AG)

2. Could you go to for help in a crisis? (PS)

3. Say something nice to you when you do something good? (ES)

4. Would say something to you if something in your life wasn’t going right? (AG)

5. Could you go to if you need advice about personal problems? (PS)

6. Could you talk to if you are upset or mad about something? (ES)

[Response options]

None

One

Two or Three

More than Three

Sense of Belonging at the Club Scale

We are interested in your opinion about the Boys & Girls Club. How much do you agree or disagree with

each of the following statements?

1. I feel like I belong here.

2. People listen to me here.

3. I feel like my ideas count here.

[Response options]

Strongly Agree

Agree

Disagree

Strongly Disagree

3 The youth version of the survey had an additional question that was not included in the teen version of the survey. The additional question was not included in the scale analyses for consistency.

Prepared by OMNI Institute XXXI

Emotional Safety at the Club Scale

We are interested in your opinion about the Boys & Girls Club. How much do you agree or disagree with

each of the following statements?

1. This Boys & Girls Club has rules for how people are supposed to treat each other.

2. I feel respected by staff at the Boys & Girls Club.

3. I feel respected by other kids at the Boys & Girls Club.

4. People make sure rules about how we treat each other are followed.

[Response options]

Strongly Agree

Agree

Disagree

Strongly Disagree

Fun at the Club Scale

Think about your experiences at the Boys & Girls Club. How true are the following statements?

1. At the club, I have a good time.

2. I enjoy coming to the Boys & Girls Club.

3. I have more fun at the Boys & Girls Club than other places I spend time.

[Response options]

Very true

Sort of true

Not very true

Not at all true

Recognition Scale

Think about the relationships you have with the staff at the Boys & Girls Club. The following questions are

about these relationships and how you are treated by staff. How true is each statement? At the club...

1. Staff reward me when I do a good job.

2. Staff notice when I try hard.

3. Staff let others know when I do a good job.

[Response options]

Very true

Sort of true

Not very true

Not at all true

Prepared by OMNI Institute XXXII

Staff Expectations Scale

Think about the relationships you have with the staff at the Boys & Girls Club. The following questions are

about these relationships and how you are treated by staff. How true is each statement?

At the club...

1. There is an adult who always wants me to do my best.

2. There is an adult who believes that I will be a success.

3. There is an adult who expects me to follow the rules.

[Response options]

Very true

Sort of true

Not very true

Not at all true

Physical Safety at the Club Scale

How often do these things happen at the Boys & Girls Club?

1. I feel safe when I am at the Boys & Girls Club.

2. If someone wanted to hurt me or beat me up here, someone at the Boys & Girls Club would stop

them.

[Response options]

All of the time

Most of the time

Sometimes

Never

Compared to when you are hanging out somewhere else, how safe do you feel when you are at the

Boys & Girls Club?

[Response options]

A lot more safe compared to hanging out somewhere else

A little more safe compared to hanging out somewhere else

Just as safe as hanging out somewhere else

A little less safe compared to hanging out somewhere else

A lot less safe compared to hanging out somewhere else

Prepared by OMNI Institute XXXIII

Safety by Club Location Scale

During the past year, did you ever stay away from any of the following places because you thought someone might attack or harm you there?

1. The entrance into the Boys & Girls Club

2. Any hallways in the Boys & Girls Club

3. Any restrooms in the Boys & Girls Club

4. Other places at the Boys & Girls Club

5. The shortest route to the Boys & Girls Club

[Response options]

Yes

No

Conflict Resolution Skills Scale4

For each of the following statements, choose the answer that best describes you.

1. When I have problems with other people my age, I push or hit the other person so that it doesn’t

happen again.

2. When I have problems with other people my age, I yell at them.

3. When other people my age try to hit or push me around, I fight back.

[Response options]

Very true

Sort of true

Not very true

Not at all true

Number of Times in a Physical Fight Item (T)

During the past 12 months, how many times were you in physical fight?

[Response options]

0 times

1 time

2 or 3 times

4 or 5 times

6 or 7 times

8 or 9 times

10 or 11 times

12 or more times

4 Two questions from the original survey were dropped from the scale because they did not have high internal reliability with the other items in the scale in 2012 and the scales were kept consistent from year to year to allow for analyses over time.

Prepared by OMNI Institute XXXIV

Number of Arrests Item (T)

How many times, if any, have you been arrested in the past 12 months?

[Response options]

None

One

Two or more times

Leadership Skills Scale

The next few questions are about working with other people your age. How much do you agree or

disagree with each of the following statements?

1. Once I know what needs to be done, I am good at planning how to do it.

2. I am pretty good at organizing a team of kids to do a project.

3. If I’m the leader of a group, I make sure that everyone in the group feels important.

4. I feel like I can stand up for what I think is right, even if my friends disagree.

[Response options]

Strongly Agree

Agree

Disagree

Strongly Disagree

Civic Responsibility-Concern for Others Scale

How much do you agree or disagree with each of the following statements?

1. I try to help when I see people in need.

2. When I make a decision, I try to think about how other people will be affected.

3. I want to help when I see someone having a problem.

[Response options]

Strongly Agree

Agree

Disagree

Strongly Disagree

Civic Responsibility- Concern for Community Scale

Please choose one answer for each of the following statements.

1. I spend time on projects with other people to help the community.

2. I believe that I can make a difference in my community.

3. I have done things to help people in my community.

[Response options]

Strongly Agree

Agree

Disagree

Strongly Disagree

Prepared by OMNI Institute XXXV

Expectations for School Success Item (Y/T)

How far in school do you think you will get?

[Response options for Youth]

Grade 8

Some high school

High school

College

[Response options for Teen]

Some high school

High school graduation

Finished vocational or trade school

1 or 2 years of college

4 year college degree

Master’s degree, Ph.D., M.D., or equivalent

Employment Item (T)

Did you have a full-time or part-time job for pay last summer?


This does not apply to me. I am not old enough to work.

No – I did not look for a job last summer.

No – I looked for a job last summer but could not find one.

Yes – I had a part-time job last summer.

Yes – I had a full-time job last summer.

Community Service Item (Y)

In the last year, how often have you helped out at the Boys & Girls Club, for example, helped out with

snack time, taken care of equipment, worked at a special event like a party, etc.?


Never

About once a year

About once a month

About once every two weeks

About once a week or more

Prepared by OMNI Institute XXXVI

Community Service (T)

In the last year, how often have you...

1. Volunteered in your school, neighborhood, or out in the community?

2. Helped out at the Boys & Girls Club, for example, tutoring younger members, leading activities,

planning events at the Club, etc.

[Response options]

Never

About once a year

About once a month

About once every two weeks

About once a week or more

Exercise (Y)

In the past 7 days (week), on how many days did you exercise, dance, or play sports for at least an hour?

[Response options]

0 days

1 day

2 days

3 days

4 days

5 days

6 days

7 days

Exercise (T)

During the past 7 days, on how many days were you physically active for a total of at least 60 minutes per

day? (Add up all the time you spend in any kind of physical activity that increases your heart rate and

makes you breathe hard some of the time.)

[Response options]

0 days

1 day

2 days

3 days

4 days

5 days

6 days

7 days

Prepared by OMNI Institute XXXVII

Healthy eating

During the past 24 hours (yesterday) how many times did you...?

1. Drink milk or eat yogurt? (In any form, including in cereal)

2. Drink 100% fruit juices, such as orange, apple or grape? (Do not count punch, Kool-Aid, sports

drinks, and fruit-flavored drinks)

3. Eat fruits? (Do not count fruit juice)

4. Eat vegetables? (Include salads and non-fried potatoes)

5. Drink water? (Such as from a glass, bottle or fountain)5

[Response options]

0 times

1 time

2 times

3 times

4 times

5 or more times

Unhealthy eating

During the past 24 hours (yesterday) how many times did you...?

1. Drink soda pop?

2. Eat French fries, potato chips, or other fried potatoes?

[Response options]

0 times

1 time

2 times

3 times

4 times

5 or more times

5 This question was added to the 2014 survey but was not included in the scale to allow for consistency across years.

Prepared by OMNI Institute XXXVIII

Tobacco Use (T)

1. Have you ever tried cigarette smoking, even one or two puffs?

[Response options]

No

Yes

2. During the past 30 days, on how many days did you smoke cigarettes?

[Response options]

0 days

1 or 2 days

3 to 5 days

6 to 9 days

10 to 19 days

20 to 29 days

All 30 days

Alcohol Use (T)

1. During your life, on how many days have you had at least one drink of alcohol?

[Response options]

0 days

1 or 2 days

3 to 9 days

10 to 19 days

20 to 39 days

40 to 99 days

100 or more days

2. During the past 30 days, on how many days did you have at least one drink of alcohol?

[Response options]

0 days

1 or 2 days

3 to 5 days

6 to 9 days

10 to 19 days

20 to 29 days

All 30 days

Prepared by OMNI Institute XXXIX

Marijuana Use (T)

1. During your life, how many times have you used marijuana?

[Response options]

0 times

1 or 2 times

3 to 9 times

10 to 19 times

20 to 39 times

40 to 99 times

100 or more times

2. During the past 30 days, how many times have you used marijuana?

[Response options]

0 times

1 or 2 times

3 to 9 times

10 to 19 times

20 to 39 times

40 or more times

Sexual Activity (T)

1. Have you ever had sexual intercourse?

[Response options]

No

Yes

2. How old were you when you had sexual intercourse for the first time?

[Response options]

I have never had sexual intercourse.

11 years old or younger

12 years old

13 years old

14 years old

15 years old

16 years old

17 years old or older

Prepared by OMNI Institute XL

3. The last time you had sexual intercourse, did you or your partner use a condom?6

[Response options]

I have never had sexual intercourse.

No

Yes

6 This item was only administered to a subset of survey respondents in 2014 as part of a separate module, unlike in previous years.

Prepared by OMNI Institute XLI

Appendix II: SAYO Scales and Items

Program Experiences Scales

Choice/Autonomy Scale

Challenge Scale

Prepared by OMNI Institute XLII

Supportive Adult Scale

Sense of Competence Scales

Sense of Competency as a Learner Scale

Prepared by OMNI Institute XLIII

Sense of Competency as a Reader Scale

Sense of Competence Socially Scale

Prepared by OMNI Institute XLIV

Future Planning and Expectations Scales (Grades 6-12 only)

Future Planning - My Actions Scale

Future Expectations Scale

Prepared by OMNI Institute XLV

Appendix III: Detailed Data Tables Table 21: 2015 Survey Outcomes by Club Attendance Frequency

Survey Item(s) (Youth/Teen Only) Range of Responses

N Moderately Frequent

Attenders N

Frequent Attenders

Sig.

Support at the Club 1-4 92 3.14 344 3.19 p=.55

Feeling of Belonging at the Club 1-4 94 3.22 347 3.25 p=.69

Feeling Emotionally Safe at the Club 1-4 94 3.29 346 3.31 p=.83

Feeling Physically Safe at the Club 1-4.5 95 3.61 352 3.71 p=.20

Number of Club Areas Avoided 0-5 95 0.91 352 1.21 p=.13

Fun at the Club 1-4 94 3.41 349 3.46 p=.51

Validation and Recognition 1-4 93 3.28 347 3.39 p=.22

Expectations from Adults for Success 1-4 94 3.63 347 3.70 p=.26

Conflict Resolution Skills 1-4 94 2.64 342 2.59 p=.69

Leadership Skills† 1-4 93 1.68 340 1.57 p=.08^

Community Service at the Club (Y) 1-5 61 3.05 230 3.20 p=.45

Community Service at the Club (T) 1-5 31 2.87 115 3.17 p=.87

Community Service at the School (T) 1-5 32 2.50 116 2.69 p=.81

Importance of School 1-5 95 1.53 347 1.48 p=.64

Healthy Eating 0-5 94 2.73 345 2.87 p=.33

Unhealthy Eating

0-5 90 1.77 334 1.62 p=.38

**p < .01, *p < .05, ˄p < .10; † Lower scores indicate stronger leadership skills.

Table 22: 2015 School Outcomes by Club Attendance Frequency

Range of Responses

N Moderately

Frequent Attenders N

Frequent Attenders

Sig.

Average School Attendance 0.54-1.00 161 0.92 430 0.93 p=.03*

Behavior Incidents 0-4 161 0.14 430 0.07 p=.09^

Suspensions 1-11 27 2.11 54 1.39 p=.04*

Weighted GPA 0-4.91 54 2.42 122 2.41 p=.92

English Proficiency 1-5 145 2.30 412 2.30 p=.96

Math Proficiency kPProficiency†Proficiency†

1-5 143 2.16 407 2.18 p=.06^

**p < .01, *p < .05, ˄p < .10.

Prepared by OMNI Institute XLVI

Table 23: 2015 Survey Outcomes by Sexual Health Participation

Survey Item(s) (Youth/Teen Only) Range of Responses N

Non-Sexual Health

N Sexual Health

Sig.

Support at the Club 1-4 794 3.08 160 3.07 p=.39

Feeling of Belonging at the Club 1-4 811 3.23 163 3.14 p=.02*

Feeling Emotionally Safe at the Club 1-4 805 3.32 163 3.12 p<.001**

Feeling Physically Safe at the Club 1-4.50 835 3.66 167 3.63 p=.13 Number of Club Areas Avoided 0-5 836 1.17 167 0.90 p=.86

Fun at the Club 1-4 815 3.45 165 3.41 p=.22

Validation and Recognition 1-4 814 3.30 164 3.29 p=.68

Expectations from Adults for Success 1-4 814 3.68 164 3.67 p=.72 Conflict Resolution Skills 1-4 804 2.70 162 2.69 p=.84

Leadership Skills† 1-4 804 1.63 163 1.64 p=.86

Importance of School 1-5 822 1.47 165 1.60 p=.83

Community Service at the Club (Y) 1-5 584 3.01 96 3.50 p=.01** Community Service at the Club (T) 1-5 228 3.04 71 3.48 p=.14

Community Service at the School (T) 1-5 234 2.67 70 2.87 p=.49

Healthy Eating 0-5 804 2.79 161 2.52 p=.07^

Unhealthy Eating 0-5 776 1.50 155 1.48 p=.74 **p < .01, *p < .05, ˄p < .10; † Lower scores indicate stronger leadership skills.

Table 24: 2015 School Outcomes by Sexual Health Participation

Range of Responses N

Non-Sexual Health

N Sexual Health

Sig.

Average School Attendance 0.54-1 504 0.94 87 0.91 p=.004**

Behavior Incidents 0-4 504 0.08 87 0.13 p=.56

Suspensions 1-11 70 1.61 11 1.73 p=.90

Weighted GPA 0-4.91 130 2.48 46 2.22 p=.11

English Proficiency 1-5 477 2.31 80 2.25 p=.97

Math Proficiency kPProficiency†Proficiency†

1-5 472 2.20 78 2.05 p=.49

**p < .01, *p < .05, ˄p < .10.

Prepared by OMNI Institute XLVII

Table 25: Differences between Club Members and Nonmembers in 2015 School Outcomes

Range of Responses N Non-members N Club members Sig.

Average School Attendance 0-1 1210 0.90 591 0.93 p<.001**

Behavior Incidents 0-5 1325 0.10 591 0.09 p=.72

Suspensions 1-11 137 1.77 81 1.63 p=.50

Weighted GPA 0-5.06 384 2.35 176 2.41 p=.48

English Proficiency 1-5 1007 2.40 557 2.30 p=.08^

Math Proficiency 1-5 1002 2.28 550 2.18 p=.048*

**p < .01, *p < .05, ˄p < .10.

Table 26: Two-Year Change in Survey Outcomes, 2014 to 2015

Statistical Significance of Two-Year Change in Survey Outcomes

Survey Item(s) (Youth/Teen Only) Range of Responses

N 2014 2015 Sig.

Support at the Club 1-4 62 3.17 3.20 p=.82

Feeling of Belonging at the Club 1-4 63 3.31 3.19 p=.35

Feeling Emotionally Safe at the Club 1-4 62 3.23 3.26 p=.80

Feeling Physically Safe at the Club 1.33-4.50 66 3.70 3.69 p=.96

Number of Club Areas Avoided 0-5 66 0.74 0.79 p=.85

Fun at the Club 1.33-4 63 3.50 3.53 p=.71

Validation and Recognition 1-4 65 3.35 3.36 p=.88

Expectations from Adults for Success 1.67-4 64 3.78 3.76 p=.74 Conflict Resolution Skills 1-4 63 2.75 2.56 p=.12

Leadership Skills† 1-3.25 61 1.62 1.58 p=.59

Community Service at the Club (Y) 1-5 33 3.18 3.45 p=.20

Community Service at the Club (T) 1-5 21 3.90 3.48 p=.17

Community Service at the School (T) 1-5 21 2.95 2.81 p=.65

Healthy Eating 0-5 64 2.82 2.63 p=.30

Unhealthy Eating 0-5 61 1.66 1.57 p=.67

**p < .01, *p < .05, ˄p < .10; † Lower scores indicate stronger leadership skills.

Table 27: Two-Year Change in School Outcomes, 2014 to 2015

Range of Responses N 2014 2015 Sig.

Weighted GPA 0.51-4.91 18 2.33 2.44 p=.58

Average Attendance 0.66-1.00 66 0.94 0.93 p=.03*

Behavior Incidents 0-1 66 0.05 0.05 N/A

Suspensions 0-5 2 3.50 2.50 p=.80

**p < .01, *p < .05, ˄p < .10.


XLVIII

Table 28: Three-Year Change in School Outcomes, 2012-13 to 2014-15

Range of Responses N 2012-13 2013-14 2014-15 Sig.

Average School Attendance 0.54-1.00 25 0.94 0.94 0.90 p=.052^

**p < .01, *p < .05, ˄p < .10

Table 29: Correlations between 2015 Survey Outcomes and 2014-15 School Outcomes

Attendance GPA

PARCC Math

Proficiency

PARCC English

Proficiency

Support at the Club -.010 .050 -.013 -.031

Belonging at the Club

-.044 .020 -.023 -.031

Emotional Safety -.013 .004 -.034 .005

Physical Safety -.047 .060 -.063 -.022

Fun at the Club -.020 .080 -.162** -.104*

Validation & Recognition

-.002 .071 -.221** -.175**

Expectations from Adults

.029 .130 -.038 .009

Conflict Resolution Skills

.170** .303** .260** .255**

Importance of School

-.187** .005 .033 .008

Leadership Skills .051 -.137 .040 .004

Healthy Eating .066 -.007 -.094^ -.062

Unhealthy Eating -.110* -.259** -.222** -.185**

Teen Volunteering (School/Community)

-.069 .071 .040 .028

Teen Volunteering (Club)

-.113 .133 .006 .016

Youth Volunteering (Club)

-.155** .018 .007 .035

Club Areas Avoided .068 -.016 -.132** -.205**

Notes: **p < .01, *p < .05, ˄p < .10. A correlation is a number that expresses the strength of the relationship between two variables. Correlations can range from -1 to 1 with values further from 0 signifying a stronger relationship between the two variables. A positive correlation indicates that higher values on one outcome are associated with higher values on the other outcome. A negative correlation (prefaced with the “-“symbol) indicates that higher values on one outcome are associated with lower values of the other outcome. A correlation value of or near zero indicates that there is no relationship between the two variables. Correlations indicate associations, but do not indicate a causal relationship between variables.

2014 – 2015 evaluation of the boys & girls clubs of metro ... · 2014 – 2015 evaluation of...

Documents