Statistical Analysis of the Safety Impacts

of Digital On-Premise Signs

H. Gene Hawkins, Jr., Ph.D., P.E.Texas A&M University

Presented at theNational Signage Research and Education

ConferenceCincinnati, Ohio; October 11, 2012

• Information, information, + more information– Traffic signs, in-vehicle displays, GPS

navigation, billboards, business signs, etc.

• As more information is presented, there is more competition for attention

The Problem

Businesses• Business owners want customers:– Loyal/repeat customers (familiar/old)

• Know the way, just need reminder– New customers (unfamiliar/new)

• Need to find the way– Both customer groups rely upon signs for

navigating to the business• Business owners also rely upon signs

for advertising and marketing

Types of Signs• Traffic signs– In the roadway right-of-way– Regulate, warn, and guide traffic

• On-premise signs– On business building or property– Get traffic to business

• Off-premise signs– Away from business– Generate interest in business

Competition for Attention • Drivers deal with:– Vehicle control

• Steering + brakes– Cockpit controls

• Environment• Music• GPS

– Navigation• Short-term• Long-term

• Other challenges– Increased traffic, higher speeds, more

controls

– Distractions• Phones (talking)• Phones (texting)• Phones (email)• Reading (books)• Listening (books)• Eating• Drinking• Make-up• Passengers• Other vehicles

Recent Example• From Bryan-College Station Eagle

How to Attract Attention

• Make it:– Bigger– Brighter– Add motion/movement– Increase contrast

• Applies to all visual stimuli– Traffic signs, business signs, people,

objects, etc.

Signage Advances• Recent technologies allow:– Electronic signs or panels– Custom messages– Animation– Video

• LEDs and other advancements provide more features at lower costs

• Potential concern that electronic displays increase driver distraction

20122003

The Challenge• Many issues combine to create a challenge:

– Advanced sign technologies are relatively new– Have spread rapidly– While traffic sign research is sponsored by

public agencies, there is not comparable research program for business signs

• Concerns over traffic safety– Base issue: Do electronic message signs create

a distraction problem that increases crash risk?– Result: Has led some local agencies to establish

sign codes that are based on opinion more than scientific fact

Our Research Study• Goal:–Determine if the installation of digital

on-premise signs has an impact on traffic safety in the area around such signs

• Highlights:– Scientific procedure– Robust crash dataset– Large sample size of sign locations– Advanced statistical analysis methods

Digital Sign Definition• Target: on-premise digital sign– Located on business property– Sign uses electrical display– Provides changeable message

Related Research• Limited research conducted on business signs• On-premise signs:

– Mace (2001): synthesis of literature• Hypothesized that distraction potential of signs could

compromise safety• Hypothesized benefits as navigational aid• No data collection to support or refute claims

– Wachtel (2009): synthesis suggested on-premise signs affect safety more than off-premise signs • Because locations and elevations of on-premise signs

might be closer to the road users• Angles of on-premise signs may be out of vision core and

require extreme head movements• Conclusions of both are based on educated

judgment rather than scientific analysis

Off-Premise Signs• FHWA study by Molino et al.

(2009)–Meta analysis of 32 previous studies– Focus on billboards–Most, but not all, of previous research

shows negative safety impact• Although safety issues not

resolved, there is more analysis of off-premise signs than on-premise

Knowledge Gap• Little knowledge about safety impacts

of on-premise signs–What is known is not based on

detailed scientific analysis– Inconsistent findings–Weaknesses

• Inadequate sample sizes• Inappropriate statistical analysis methods• Short time frames for analysis (before & after)

• At present, cannot define safety impacts of digital signs

Research Approach• Highlights:– Collect sign data• Specific to on-premise digital signs• Installation date and location are critical

– Collect crash data at locations where signs are located• Crash information: date, location, type,

etc• Need several years of data

–Develop statistical analysis procedure– Perform safety analysis

Sign Data Acquisition• Required significant effort• Initial attempts not successful– Asking for too much

• Refined request based on crash data dates and states– Installed in 2006 or 2007

• Insures adequate before and after periods– Located in California, Illinois, Maine,

Minnesota, North Carolina, Ohio, or Washington• These are states with crash data

Sign Data Sets• Sign datasets acquired from two

companies– #1: 2,953 sites with 27 variables• Variables: date, address, cross-street,

road/traffic information, etc.– Road/traffic information not used

– #2: 63 sites with 10 variables• Sign locations had to be confirmed

through Google Earth/Map to be usable

Sign Data Processing• Raw sign data required significant

processing to be useable• Sites eliminated due to:– Installed before 2006 or after 2007

• These limits provided sufficient date in both before and after analysis periods

• Year of installation not included in analysis– Location could not be confirmed in

Google Earth or Google Maps (Street View)• Analysis area defined as

within 0.1 mile of target sign

Crash Data Background

• Comprehensive crash data is limited and hard to obtain– Belongs to agencies– Protected information for liability reasons– Expensive to maintain

• National databases– FARS: Fatal Accident Reporting System• NHTSA owned – no info on crash location

– HSIS: Highway Safety Information System• FHWA owned – significant related information

HSIS Crash Data• HSIS provided the best means of

performing a detailed national analysis of crash impacts due to digital signs

• Contains crash, roadway, and vehicle information

• Operated/maintained by FHWA• Widely used for safety research • Multi-state data– California, Illinois, Maine, Minnesota, North

Carolina, Ohio, and Washington– We chose to focus on CA, NC, OH, and WA due

to limited sign locations in IL, ME, and MN

HSIS File Types• Crash files:

– Location, date, time, light, weather conditions, severity, number of related vehicles, collision type

• Driver and vehicle files:– Driver gender, age, contributing factor (possible

casual factor), vehicle type• Road and traffic files :

– Traffic data – Average Daily Traffic (ADT), speed limit – Road data – number of lanes, lane and median width,

shoulder width and type, rural or urban designation, and functional classifications.

• All files are linked to provide for robust analysis– Example: crash frequency and traffic volume can be

combined to present a crash rate at a location

Data Analysis Steps• General steps– Confirm location of signs– Convert sign location to crash

location– Evaluate site qualification factors– Conduct statistical analysis

• Each is explained in upcoming slides

Confirm Sign Location• Use Google Map and Google Earth– Verify location of each target sign• Used street address provided in sign data

set– Confirm that it is a digital sign– Confirm that is it an on-premise sign– Confirm still in place (as of the date of

the Google Street View image)–Measure milepost from county

boundary• Used to link location to crash data

Confirm Sign Location• Labor intensive and time consuming

activity• Used student workers to process data

Combine Sign and Crash Data

• Sign and crash data use different location systems– Sign data based on street address– Crash data based on route and

milepost• Convert sign location to route and

milepost format and combine with crash location data set

Site Qualification Factors

• We only retained the sign sites satisfying the following conditions:– Located in CA, NC, OH, or WA– Installed between 2006-2007– Located on the major roads –With at least one crash record in the

before or after period

Sign Data Yield Rates• After processing sign and crash data,

the number of sites usable for analysis was a fraction of initial number– Initial sign data sets: 2,953 + 63 = 3,016

–Overall yield rate = 126/3016 = 4.2%

State CA NC OH WA AllInitial 86 249 372 413 1,120Install Date 2006-2007 27 94 178 159 458Digital+Major Rd+Crashes 5 28 60 33 126Yield Rate 5.8% 11.2% 17.2% 8.0% 11.3%

Statistical Analysis Options

• Options for analyzing safety impacts:– Before-after study

• Crashes in the period before improvement compared to crashes in the period after the improvement

• Provides more direct evaluation– Cross-sectional study

• Crashes on a facilities with the improvement compared to crashes on similar facilities without the improvement

• Different facilities rarely identical in all features

• We chose to use before-after study

Types of Before-After • Naïve before-after study– Simple comparison– Results may be influenced by

factors not accounted for– Not a preferred analysis method

• Before-after study with control group– Control group helps to account for external

influences that could affect results– Requires additional data for control locations– Difficult to identify appropriate control locations

with same characteristics• Before-after study using the empirical Bayes

(EB) method

Empirical Bayesian Method• Recommended as preferred method in Highway

Safety Manual• Combines short-term observed crash numbers

with crash prediction model data to obtain a more accurate estimation of the long-term crash mean– Example: a new driver has no crashes during first

year of driving– Typical new driver has 0.08 crashes per year– Not reasonable to expect 0 or 0.08 crashes in

second year– EB would provide an estimate that is a mixture of

these two values by considering safety of a specific segment and safety of a typically similar road

EB Analysis Formula• General form:

• Where:

CrashMeanˆCrashMean

CrashFrequencyˆWeight (CrashPrediction) (1 Weight) CrashFrequency

afterEB

after

before

beforeEB

1(1 )

Weight Weight factor of theEB

overdispersion parameter crash prediction number

Crash Prediction estimated crash number based on the HSM crash prediction models (control data)

Preliminary Study Results• Interpretation factors• All crashes– For entire sample– For each state

• Single and multiple vehicle crashes– For entire sample– For each state

• These results are preliminary– Research in final stages of completion

Interpreting Results• Using the EB method, there is no

statistically significant change in crashes if the confidence interval for EB contains 1

Preliminary Results for All Crash Types

28 60 33 121No. of SitesSample Size

NC OH WA ALL0.8

0.9

1.0

1.1

1.2

1.3

1.4

1.5higher boundѲlower bound

NC OH WA ALL0.0

0.5

1.0

1.5

2.0

2.5

higher bound Ѳ

lower bound

Preliminary Results for Single Vehicle Crashes

23 48 25 96

No. of SitesSample Size

Preliminary Results for Multiple Vehicle Crashes

27 56 33 116

No. of SitesSample Size

NC OH WA ALL0.8

0.9

1.0

1.1

1.2

1.3

1.4

1.5

1.6

higher bound

Ѳ

lower bound

Preliminary Conclusions

• For the entire sample:– We did not find a statistically significant

impact between the installation of digital signs and an increase in crashes within 0.1 mile of the signs’ locations

• For multiple-vehicle crashes– We found the same result

• For single-vehicle crashes– We found the same result except for California,

which is likely due to the smaller sample size and the resulting decrease in statistical certainty (increased probability of error)