national evaluation enforcing underage drinking laws program eudl national leadership conference...
TRANSCRIPT
National Evaluation National Evaluation Enforcing Underage Drinking Laws ProgramEnforcing Underage Drinking Laws Program
EUDL National Leadership ConferenceEUDL National Leadership ConferenceDallas, TexasDallas, Texas
September 19, 2002September 19, 2002
Funding provided by the Office of Juvenile Justice and Delinquency Prevention
Acknowledgements
All 50 state EUDL coordinators
All Key Actor Survey participants
All Law Enforcement Agency Survey respondents
OJJDP, PIRE, and USC
Evaluation DesignEvaluation Design
Process Evaluation
Key Actor Survey
Case Studies
Local Project Director Survey
50 States 50 States + D.C. 50 States + D.C.
1999 2000 2001
6 States 6 States
Impact Evaluation
Law Enforcement Agency and Youth Surveys
52 Intervention 52 Comparison
17 Intervention 17 Comparison
17 Intervention 17 Comparison
52 Intervention 52 Comparison
52 Intervention 52 Comparison
First Round of Discretionary Grants
Second Round of Discretionary Grants
3 States (BP)
216 Projects
Evaluation Design (cont.)Evaluation Design (cont.)
Process Evaluation
Key Actor Survey
Local Project Director Survey
50 States + D.C. 50 States + D.C. 50 States + D.C.
2002 2003 2004
Impact Evaluation
Law Enforcement Agency and Youth Surveys
17 Intervention 17 Comparison
Second Round of Discretionary Grants
Fourth Round of Discretionary Grants
36 Projects
36 Intervention 36 Comparison
36 Intervention 36 Comparison
36 Intervention 36 Comparison
Themes
Galvanized Activity and New Relationships
Increased Enforcement
Youth Behavior – Any Change?
Best and Most Promising Practices
State Government Agencies “Very Involved” in EUDL
3431
2624 23
0
10
20
30
40
50
Highway Safety State Police Substance AbusePrev. &
Treatment
ABC Juvenile Justice
Year 3
N = 49
State Government Agencies “Very Involved” in EUDL (cont.)
23
13 1311 10
0
10
20
30
40
50
Governor'sOffice
AttorneyGeneral
Public Health HumanServices
Education
Year 3
N = 49
Local Organizations “Very Involved” in EUDL
33
1713
7 6
0
10
20
30
40
50
LawEnforcement
YouthOrganizations
SubstanceAbuse Prev. &
Treatment
High Schools Local HealthDepts.
Year 3
N = 49
Other Groups “Very Involved”in EUDL
24
108 8
0
10
20
30
40
50
Citizens' Groups Judges/Judicial Org Alcohol Merchants State Legislators
Year 3
N = 49
Types of Interventions(No. of States “Very Involved”)
42%
55%
38%40%40%
82%
64%
48%
80%
52%
86%
28%
38%
88%
48%
70%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Coordination Public Education Enforcement Policy Change
Prior to EUDL
Year 1
Year 2
Year 3
N = 40-50
Additional Interventions Additional Interventions (No. of States “Very Involved”)(No. of States “Very Involved”)
23
11
21
27 27
12
0
10
20
30
40
50
Youth Involvement Targeting Colleges Programs for YouthOffenders
Nu
mb
er o
f S
tate
s
2000
2001
(N= 50)(N= 50)
Interventions Funded with Local EUDL Grants
2001 Local Project Director Survey
73%
50%
64%74%
76%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Enforcement YouthInvolvement
Education CoalitionBuilding
Policy Change
(N = 211)
Local Agency Focus on Underage DrinkingBefore and After Receiving EUDL Funding
41.7%36.0%
79.8%75.2%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Block Grant Discretionary Grant
Per
cent
age
(%)
Involvement prior toEUDL
Involvement sinceEUDL
(N = 72 – 125)
Perceived Obstacles
36
26
19
12 11 10
30
36
12
20
14 14
27
32
1
16
22
10
0
10
20
30
40
50
InsufficientResources
Not Problem forPublic
Lack ofConsensus
Not Problem forState Decision
Makers
IndustryOppostion
Not Problem forLaw Enforcment
Year 1
Year 2
Year 3
N = 50
Themes
Galvanized Activity and New Relationships
Increased Enforcement
Youth Behavior – Any Change?
Best and Most Promising Practices
Enforcement Activities Enforcement Activities (No. of States “Very Involved”)(No. of States “Very Involved”)
Compliance Checks - 35 States
Cops in Shops – 22 States
Shoulder Tap program – 16 States
N = 49-50N = 49-50
Target of Enforcement Intervention (No. of States “Very Involved”)
41
34
2932
22
31
36
30
21
0
10
20
30
40
50
Merchants Underage Adult/Social
Year 1
Year 2
Year 3
N = 49-50
Impact Evaluation: Analysis Design
1999 Data Collection
Year 1 Year 2 Year 3
2000 Data Collection
2001 Data Collection
2000 Data Collection
2001 Data Collection
Round 1 Communities (104)
Round 2 Communities (38)
2002 Data Collection
Median Number of Compliance ChecksMedian Number of Compliance Checks
20.020.0
24.1
14.0
0
5
10
15
20
25
30
35
40
45
50
Num
eric
al V
alue
Comparison
Intervention
P-value = 0.030
N (Comparison) = 76
N (Intervention) = 105
Year 1 Year 2
Median Number of “Cops in Shops” Median Number of “Cops in Shops” OperationsOperations
6.0
18.0
25.0
10.0
0
5
10
15
20
25
30
35
40
45
50
Comparison
Intervention
P-value = 0.005
N (Comparison) = 39
N (Intervention) = 68
Year 1 Year 2
Median Number of Arrests for Median Number of Arrests for Purchase, Possession, or Use of Purchase, Possession, or Use of
AlcoholAlcohol
45
8 11
20
0
5
10
15
20
25
30
35
40
45
50
Comparison
Intervention
P-value = 0.049
N (Comparison) = 88
N (Intervention) = 81
Year 1 Year 2
Other Enforcement Findings: No Statistically Significant Changes
Shoulder Tap ProgramMonitoring/patrolling parking lotsTracing alcohol sourceCitations/arrests for false IDCitations/fines/license suspension of
businesses
Percentage of Agencies Reporting Strong Percentage of Agencies Reporting Strong Local Support for Underage Drinking Local Support for Underage Drinking
EffortsEfforts
29%
32%
36%
45%
20%
25%
30%
35%
40%
45%
50%
Comparison
Intervention
P-value = 0.610
Odds Ratio = 0.82
N (Comparison) = 200
N (Intervention) = 207
Year 1 Year 2
Themes
Galvanized Activity and New Relationships
Increased Enforcement
Youth Behavior – Any Change?
Best and Most Promising Practices
Alcohol Consumption in the Past Alcohol Consumption in the Past 30 Days30 Days
36%40%36%
41%
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
50%
Comparison
Intervention
P-value = 0.662
Odds Ratio = 0.94
N (Comparison) = 2410
N (Intervention) = 2370
Year 1 Year 2
Other Youth Behavior Findings: No Statistically Significant Changes
Past 7 day alcohol consumptionBinge drinkingDrinking and drivingAttempts to purchase alcoholPerception of alcohol use among
peersConsequences of alcohol use
Why do Program Evaluations Seldom Detect an Impact on Behaviors?
Delayed, Incomplete, or Failed Program Implementation Flawed Program Theory Short Intervention Period Secular Trends Competing Programs Non-comparable Intervention and Comparison Groups Measurement Error Use of Wrong Measures Inadequate Statistical Power
Adapted from Lipsey and Cordray, 2000
Themes
Galvanized Activity and New Relationships
Increased Enforcement
Youth Behavior - No Solid Evidence of Change
Best and Most Promising Practices
Expert Survey Methods
Literature review identified experts in UAD research
Practitioner experts were identified using snowball sampling techniques
Web based survey N = 40Response rate = 75%
AnalysisEmpirical Evidence
Larger Quantity Smaller Quantity
Hig
hN
ot H
igh
Per
ceiv
ed E
ffec
tive
nes
s Best · ≥50% of experts perceived
practice to be highly effective and to have high quantity of empirical evidence
Promising · ≥50% of experts perceived
practice to be highly effective but did not perceive high quantity of empirical evidence
Ineffective · ≥50% of experts perceived
practice to not be highly effective and to have high quantity of empirical evidence
Questionable · ≥50% of experts perceived
practice to not be highly effective but did not perceive high quantity of empirical evidence
Compliance checks ¨ Conduct regular enforcement actions involving the use of underage
decoys who attempt to purchase alcohol
DWI Enforcement ¨ Enhance enforcement of drinking and driving laws ¨ Conduct sobriety checkpoints
Local Policy
¨ Restrict zoning (outlet locations and density)
State Policy ¨ Increase excise tax ¨ Restrict zoning (outlet locations and density) ¨ Enact .08 blood alcohol content laws for the general population
Findings: Best Practices
Findings: Promising PracticesState Policy
¨ Directly increase prices in "control" states in which prices are set by the state ¨ Enact, or strengthen existing, dram shop liability laws ¨ Restrict hours of sale ¨ Require or encourage the use of driver's license scanners ¨ Enhance driver's license to facilitate recognition of underage purchase attempts
and make license more difficult to falsify ¨ Enact and promote the use of civil penalties ¨ Ban concurrent sales of alcohol and gasoline ¨ Restrict alcohol marketing
School Policy
¨ Enact alcohol policies on college grounds and at college-sponsored events
Findings: Promising Practices
Local Policy ¨ Prohibit entry of persons under 21 into bars/nightclubs and other
"adult" locations ¨ Require or encourage the use of driver's license scanners ¨ Restrict the availability of alcohol at community festivals and other
community events ¨ Restrict alcohol industry sponsorship of public events ¨ Require conditional use permits ¨ Ban concurrent sales of alcohol and gasoline ¨ Restrict alcohol marketing
Findings: Promising Practices
DWI Enforcement Driving under the influence emphasis patrols
Other Enforcement Approaches (including training) Training of law enforcement officers to promote
better enforcement efforts Enforcement and education efforts focused on
parents and landlords who allow underage drinking parties to take place on their property
Evaluation Products
“Bulletin”Year 1, Year 2, and Year 3 ReportsDesign PaperOutcome PaperBest Practices PaperBest Practices Case StudiesPapers on Assorted Topics: LE, Sexual
Assault, Parenting
End of Leadership Conference Presentation