Effects of a comprehensive character education program on student smoking

– Findings from 4 studies

Brian R. Flay, D.Phil.Professor, Public HealthOregon State University

Corvallis, OR

Society for Research on Nicotine and Tobacco, Portland, OR, Feb 29 2008

2

Outline

I. The Positive Action program philosophy and theory

II. The Positive Action program components

III. Evaluations and results

IV.Conclusions

3

4

I. Overview of PA Program• Positive Action (PA) is a comprehensive school-based program

designed to – promote student character and positive behavior, – prevent an array of student problem behaviors, and – improve student achievement.

• PA is grounded in a broad theory of self-concept that posits – (a) students’ self-concepts and characters are determined by how

they behave and how they feel about themselves when they do various behaviors and

– (b) making positive and healthy behavioral choices results in feelings of self-worth.

• Major features of the program include: – Teacher delivered scripted PA curriculum lessons in classroom – Teacher and school staff modeling/reinforcement of “PA

behaviors” inside and outside of the classroom– School-wide activities (e.g., PA assemblies) led by principal and

PA Committee

5

Positive Action Overview

1. The core of Positive Action is contained in an underlying philosophy.

2. The philosophy is described in the Thoughts-Actions-Feelings Circle.

3. Positive Action teaches the positive actions for the whole self through six units that are contained in five program components.

4. The five completely prepared components are: 1. K–12 curriculum,2. Climate Development,3. Counselors program,4. Family program, and5. Community program.

6

You feel good about

yourself when you do positive actions.

Ref. (Reference)Cognitive Behavior Therapy

and Positive Psychology

Basic Philosophy (Theory of Action)

of the Positive Action Program & Circle

7

You feel bad about yourself when you do

negative actions.

C.F. (Reference) Depression

...and

Basic Philosophy (Theory of Action)

of the Positive Action Program & Circle

8

II. The POSITIVE ACTION Program Components

K–12 classroom curriculumover 1,200 lessons - using Teacher’s Kits (manuals and materials for each grade), classroom teachers present 15–20-minute lessons

Principal’s Kits (Elementary and Secondary)a school-climate program to promote the practice and reinforcement of positive actions in the whole school population (students and staff)

Counselor’s Kitused with selected individual students, small groups and families

Family Kit contains prepared weekly home lessons paralleling the school program along with school parent-involvement activities

Community Kitmanuals and materials that align and encourage collaboration of all the environments (schools, families and community) involved in the program

9

Positive Action Focus Units(Learning Goals)

• In the classroom curriculum and all other materials, the Positive Action content is taught through six focus units.

Unit 1: Self-Concept: What It Is, How It’s Formed, and Why It’s Important (Philosophy & Circle)

Unit 2: Physical and Intellectual Positive Actions for a Healthy Body and Mind (includes motivation to learn)

Unit 3: Social/Emotional Positive Actions for Managing Yourself Responsibly

Unit 4: Social/Emotional Positive Actions for Getting Along with Others by Treating Them the Way You Like to Be Treated (Social-Emotional Skills & Character)

Unit 5: Social/Emotional Positive Actions for Being Honest with Yourself and Others (Mental Health)

Unit 6: Social/Emotional Positive Actions for Improving Yourself Continually (Setting & Achieving Goals)

10

Logic/Theoretic Model of the Expected Effects of the Positive Action Program

Program Components Immediate Outcomes

Attitudes Toward Behaviors,

Social Normative Beliefs,

Self-Efficacy

Improved School Attendance, Gradesand Test Scores

* Improved relationships among school administrators, teachers, parents & community.* Improved classroom management.* Increased involvement of school with parents & community.

Climate Development, Family Kit, Teacher/Staff Training, K–12 Instruction Curriculum, Drug Education Supplements, Community Kit,Counseling Kit

Improved

Learning

Environment

1. Improved character/self-concept2. Learning/Study skills3. Self-Management4. Interpersonal/social skills5. Self-honesty, responsibility6. Goal setting, future orientation

PA Unit

ImprovedSocialand

CharacterDevelopme

nt

Fewer Disciplinary Problems; ReducedSubstance Use; Less Violence

Expected Effects Expected Impact

11

Elementary Curriculum Kits - Grades K–6

• K–6 Instructor’s Manuals– 140 15-minute lessons

• Role-playing• Plays• Stories• Questions• Poetry• Games

• Student activity sheets/bookletsand materials for 30 students

• Posters• Music• Reinforcement stickers• Hands-on activities

–Puppets–Flannel board characters–Games–Others

12

Drug Education Curriculum Grade 5 and Middle School

• Grade 5 Drug Education Supplement Instructor’s Manual

–18 15- to 20-minute lessons• Stories• Questions

–Student activity booklets and materials for 30 students

–Posters–Music

• Middle School Drug Education Supplement Instructor’s Manual

–30 15- to 20-minute lessons• Stories• Questions

–Student activity booklets and materials for 30 students

–Scripted play: “Escape fromthe Shadows”

–Games–Posters–Music

• Lessons are taught at the end of each unit of the regular PA curriculum.

• The kits can also stand alone.

13

Elementary Climate Kit

• Principal’s manual• Support Staff manual• Parents’ manual• Assemblies

– Certificates of Achievement– Positive Action Balloons

• Positive Action Calendar• Words-of-the-Week Cards• Reinforcement Stickers • ICU Boxes and Notes• Positive Action Tokens

• Positive Notes • Notepads• Positive Action News • Music – 27 songs

– CDs– Song books

• Posters• Video

– Overview

• Implementation Plan• Scope and Sequence

Booklet

14

Other Program Components

• Counseling Program– Counselor Kit– Lessons for individual students or small groups

• Family Program– Classes– Home Kit

• Community Program– Community messages and activities– Civic engagement– Media messages

15

III. Four studies and results

• I. Intensive Case Study

• II. Matched control study using archival data

• III. RCT in Hawaii elementary schools

• IV. RCT in Chicago Public Schools

16

• First year of PA in a rural Title I school in Northern Florida.

• Variation in level of implementation

– None or Some (7 classrooms)

– Almost All (7 classrooms)

– Every Component and Lesson (11 classrooms)

• Students, Teachers and Parents surveyed at beginning and end of year.

Study I: Intensive case study of program implement in a small rural school

17

Clear dose-response relationship between level of program implementation and student smoking in grades 4 and 5 (Χ2 =

12.6, p < .01, with no diffs. at pretest)

None Medium Total

Level of PA Implementation

0.00

0.50

1.00

1.50

2.00

Mean

Ever smoked pretest

Ever smoked posttest

0=Never, 1=Once, 2=More than onceGrade 4 and 5 Smoking by Level of PA

18

Study II: Matched control study using school-level archival data• School Report Card data to find matching control schools for the PA schools

– Poverty (as indicated by percentage of free/reduced lunches)

– Mobility rates

– Ethnic distribution

• We had data on feeder patterns for each middle and high school.

• Data on whether or not each elementary school had actively implemented PA for 4 or more years.

– 55 PA schools, 29 non-PA schools

• Calculated % of PA graduates in each middle and high school.

• Archival data included reports of disciplinary actions for cigarette smoking

• ANCOVA adjusting for school characteristics.

• Flay & Allred, Prevention Science, 2003

19

Effects of Elementary School PA on

Middle School Cigarette Smoking

Percent of students from PA schools

>75%65-75%<65%

Me

an

Inci

de

nts

of

tob

acc

o u

se (

'97

-98

& 9

8-9

9)

20

18

16

14

12

10

13

16

19

ANCOVA one-tailed p = .03

20

<15% 15-27% 27-50%

% Elementary PA

60.00

65.00

70.00

75.00

80.00

85.00M

ea

n N

um

be

r o

f to

ba

cc

o in

cid

en

ts

80.25

72.20

64.21

Effects of Elementary PA on HS Tobacco Use

ANCOVA one-tailed p = .04

21

Study III: Randomized Trial in Hawaii Elementary Schools

• Elementary schools (N = 20) on 3 Islands• 5yr Effectiveness Trial• Random assignment

– PA or Control conditions• Strata matched on multiple indicators of risk

• PA schools received:– 4yrs (2002-03 through 2005-06)– Teacher training

• Developer, Carol Allred 3-4hrs,initial yr; booster sessions 1-2hrs

• Technical Support, Project Coordinator Howard Humphries

• Proc Mixed ANOVA analyses accounting for students clustered within schools

22

Baseline Equivalence: 2000-01

No differences are close to being statistically significant except % free/reduced lunch p = .099

0

10

20

30

40

50

60

70

80

90

100E

nrol

/10

Sta

bilit

y %

Lunc

h %

Bud

get

/day

Per

Cap

/100

0

Sp

Ed

%

LEP

%

Bel

owA

vera

ge

Aab

ove

Ave

rage

Bel

owA

vera

ge

Abo

veA

vera

ge

Abs

ente

es%

Dis

cplu

nary

Ref

eral

s

Dis

p. R

's p

er10

stu

dent

s

SAT Read: % SAT Math: %

School Demographics Achievement Behavior

Program Schools

Control Schools

23

Baseline Equivalence on Ethnic Distribution

0.00

5.00

10.00

15.00

20.00

25.00

30.00

35.00

40.00

Ethnic Group

Perc

ent

Program Schools

Control Schools

All of Hawaii

No differences are close to being statistically significant. Sample is fairly representative of all HI schools

24

Fig. 13: % of 5th graders reporting substance use by condition

-2.00

0.00

2.00

4.00

6.00

8.00

10.00

12.00

14.00

16.00

Ever triedtobacco

Ever triedalcohol

Ever beendrunk

Ever triedillegal drugs

C P A

Hawaii Trial:

30% reduction

p = .012 for scale of combine substance use, accounting for nesting of students within schools

25

Hawaii program effects on absenteeism

Figure 8: Average Daily Absences by Condition

8

8.5

9

9.5

10

10.5

11

11.5

2001-02 2002-03 2003-04 2005-05

Sch

oo

l Y

ear

CONTROL PA STATE Standard

26

Hawaii dose-response relationship

Figure 19: Teacher ratings of students "Gets Along With

Others" by level of implementation and control

2.35

2.4

2.45

2.5

Hi Med Low Control

27

Study IV: Randomized Trial of PA in Chicago Public Schools

• Elementary Schools (N = 14)• 3 yr Effectiveness Trial• Random Assignment

– PA or Control Conditions• Strata matched on multiple indicators of risk

• PA Schools Received:– 3 yrs (2003-04 through 2006-07)– Teacher Training

• Developer: Dr. Carol Allred 3-4 hrs training, initial yr; booster sessions 2 hrs, subsequent yrs

• Technical Support: Chicago Project Coordinator, Vanessa Brechling

• Data Collection– Local (UIC)– Multi-site (US Dept of Education)

28

Comparability of Matched Sets of SchoolsChicago Study (No significant differences)

0

10

20

30

40

50

60

70

80

90

100

Pair

Readin

ess

% W

hite

% B

lack

% H

isp

% A

sian

Achiev

e

% A

ttten

d

% T

ruan

cy

% P

over

ty

% M

obilit

yEnr

ol

% P

aree

nt P

atici

p

Quality

Tea

cher

s

Crimes

Pe

rce

nt

ES (Program)

LS (Control)

29

Figure B: Effects of 3 years of PA on Behavior% Improvement

28.2

21.7

12.5

0 5 10 15 20 25 30

Smoking

Alcohol use

Serious violence

% Improvement

30

% Ever Used Sustances and % Reduction by Condition:Grade 5 Chicago Randomized Trial

13.2%

36.2%

10.7%

39.9%

9.4%

28.3%

5.1%

32.1%

28.2%

21.7%

52.2%

19.5%

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

Ever smoked/usdtobacco

Ever drank alcohol Ever gotten drunk Ever usedtobacco, alcohol

or drugs

Pe

rce

nt

C PA % reduction

p = .023 for scale of combined substance use,

accounting for clustering of students within schools

31

Effects on School-Level reports of misconducts and suspensions (Chicago)

In ANCOVA models predicting year 4 differences from year 1 levels and condition, differences at year 4 are marginally significant for misconducts (p = .054)

and significant for suspensions (p = .037) using one-tailed tests.

Average N of suspensions per 100 students by year and condition (77% reduction at 2006-07)

0.00

5.00

10.00

15.00

20.00

25.00

30.00

35.00

2003-04 2004-05 2005-06 2006-07

Year

PA C

Average N of misconducts per 100 students by year and condition (80% reduction at 2006-07)

0.00

5.00

10.00

15.00

20.00

25.00

30.00

35.00

40.00

45.00

2003-04 2004-05 2005-06 2006-07

Year

PA C

32

IV. Conclusions

• Implementing school-wide character education programs to address a wide range of outcomes is challenging– Limited resources of urban school systems – NCLB

• Evaluation of school-based character/social development programs is complicated by control schools implementing similar programs under “business as usual” conditions

• Clear dose-response relationships were evident in quasi-experimental evaluations

• In RCTs program effects on smoking at 5th grade were obtained after 3 or 4 years of programming

• School-level reports of misconducts and suspensions strengthen the robustness of the findings

• Time trends in outcomes and long-term follow-up studies suggest increasing effects over time

• School-wide social and character development education can be effective at:– decreasing multiple negative behaviors, including smoking

33

Why does Positive Action have such strong effects in multiple domains?

• Links all behaviors/actions to feelings, thoughts and values

• Increases awareness of social influences and correction of normative beliefs

• Emphasizes universal values and principles, and is appropriate for all ethnic and SES groups

• Is consistent with multiple theories of education, learning and behavior development and change

• Works on the multiple social ecologies in which youth live and develop– School, home, community

34

Future Research

• Investigate potential differential impacts of PA based on student gender, child risk level, etc.

• Investigate whether schools with different levels in the quality of implementation yield different “impacts”

• Examine impact of PA as student cohort progresses into upper elementary grades (grades 6-8)– Critical transitional period within emotional,

behavioral, and academic domains

35

Future Work/Needs – The bigger picture

• Larger scale trials– ICCs for attitudes (.03-.1) and behavior (.01-.05) are generally smaller

than for achievement (.15-.2)– Still need Ns of 20 or more per condition rather than 7-10

• Improved measures of integrity and dosage delivered and received– Teacher, student and observer reports– Contractual reporting systems?

• Longer term follow-ups– Effects take several years to even start emerging– Prior work suggests important long-term effects are possible

• Methods of analysis to accommodate differential implementation– Propensity scoring, CACE, instrumental variable

36

AcknowledgmentsStudies I and II were unfunded, conducted by the author at the University of Illinois at Chicago (UIC).Study III was funded by grant #R01-DA13474 from NIH/NIDA to Brian Flay, initially at UIC then at Oregon State University (OSU).The findings reported from Study IV are based on research conducted as part of the Social and Character Development (SACD) research program funded by the Institute of Education Sciences (IES), U.S. Department of Education through Grant # R215S020218 to UIC (2003-2005) and OSU (2005-2008). The SACD Consortium consists of representatives from IES, the Centers for Disease Control and Prevention (CDC), and the national evaluation contractor, Mathematica Policy Research, Inc. (MPR), and each grantee site participating in the evaluation. The content of this presentation does not necessarily reflect the views or policies of the SACD Consortium members including IES, CDC, and MPR, nor does mention of trade names, commercial products, or organizations imply endorsement by the U.S. Government. Correspondence concerning this presentation should be addressed to Brian R. Flay, D.Phil., Principle Investigator, Department of Public Health, 254 Waldo Hall, Oregon State University, Corvallis, OR 97330, brian.flay@oregonstate.edu.

37

SACD disclaimer statement:

The Social and Character Development (SACD) research program funded by the Institute of Education Sciences (IES), U.S. Department of Education includes a national evaluation study conducted by Mathematica Policy Research (MPR), and complementary research studies conducted by each grantee. The findings reported here are based on the complementary research activities carried out by Brian Flay, Oregon State University, and David L. DuBois, University of Illinois at Chicago, under the SACD program. These findings may differ from the results reported for the SACD national evaluation study. The findings presented in this conference presentation are based on a smaller sample size of children, classrooms, and teachers, utilized a different set of outcome measures, and sought to answer complementary research questions. The content of this presentation does not necessarily reflect the views or policies of the SACD Consortium including IES, CDC, and MPR, nor does mention of trade names, commercial products, or organizations imply endorsement by the U.S. Department of Education.

38

Have a Positive Action Day!