Let’s Not Forget About Accuracy

Henry E. BradyClass of 1941 Monroe Deutsch

Professor of Political Science and Public PolicyUniversity of California, Berkeley

Accuracy and Security

• Accuracy: Votes are properly recorded according to voter’s intention

• Security: Votes are not changed because of misfeasance or malfeasance – mistakes or fraud

A Word About Security

• Note:– Risk = Threat X Vulnerability

• Many discussions only consider vulnerabilities and not threats.

• But it takes both to create a risk.

Measuring Accuracy

• Residual votes:– Number of ballots minus number of votes in a race– Example: 100 ballots & 97 votes so 3 residual votes

• Residual votes are sum of:– overvotes (ballots with more than one mark for a

contest, thus invalidating the vote) and – undervotes (ballots with no mark for a contest, thus

not counting as a vote)

• Residual vote rate-- Residual votes over number of ballots-- Example: 3/100 = 3%.

Cause of Residual Votes

• Not all residual votes are “errors” – sometimes people choose not to vote in a race: They intentionally undervote or overvote.

• But for “top-of-the-ticket” contests, residual votes should be low – around ½% to 1%.

• If two jurisdictions are similar except for voting systems and one has significantly lower residual vote rates than the other, then the difference is the performance of voting systems (technology, poll workers, administration).

Table 1 – Average of Lowest Residual Vote Rate For Each County among Eight Propositions by Polling Place Voting System Types

Type of System Model Mean Over

Counties

Number of Counties

Mean weighted by

Ballots

Number of Ballots

Direct Record Electronic

Accuvote-TS 1.00% 2 0.88% 322,048

AVC Edge 2.09% 11 0.92% 1,047,960

Hart E-slate 1.12% 1 1.12% 634,588

iVotronics 1.07% 1 1.07% 35,215

Total 1.81% 15 0.98% 2,039,811

Optical Scan—Precinct Count

Accuvote-OS 0.82% 15 0.84% 1,523,657

Eagle 2.38% 3 2.82% 364,198

Total 1.08% 18 1.22% 1,887,855

Optical Scan—Central Count

Ink-A-Vote 3.52% 1 3.52% 1,661,675

M100,550,650 0.69% 7 0.64% 739,621

Mark-A-Vote 1.60% 5 1.17% 263,703

Total 1.26% 13 2.49% 2,664,999

Punch-Card Datavote 1.01% 8 0.93% 330,168

Total 1.01% 8 0.93% 330,168

Mail Mail 0.69% 2 0.83% 1,936

Total 0.69% 2 0.83% 1,936

Total All Systems

1.29% 56 1.62% 6,924,769

Comments and Caveats

• Data are from preliminary canvas• Residual rates are by polling place voting

system – but many votes are absentee• Two types of averages in the table:

– Simple county averages– Averages weighted by number of ballots

• Two counties omitted:– Monterey (used two types of systems)– Butte – Instructive story

Further comments

• Most rates less than 1.2%

• High ones due mostly to two counties:– Los Angeles – 3.5% -- Ink-A-Vote– San Francisco – 4.0% -- ES&S Optech Eagle

• Only three other small counties over 1.6%:– Sutter – 4.5% (Mark-A-Vote)– Mariposa – 6.5% (Sequoia AVC Edge)– Mono -- 8.5% (Sequoia AVC Edge)

Lowest Residual Vote Rate for Each County

8.75

8.13

7.50

6.88

6.25

5.63

5.00

4.38

3.75

3.13

2.50

1.88

1.25

.63

Figure 1: Histogram of Residual Vote Rates

for 57 California Counties 35

30

25

20

15

10

5

0

Std. Dev = .01

Mean = .01

N = 57.00

LA,SF,Sutter Mariposa Mono

What’s Going on In Los Angeles?

• Consider November 2004 data

• Strong positive relationship between:– Percent minority and residual vote

• But this seems to be the result of strong negative relationship between: – Percent high school graduates and residual

vote

Pres. Residual Rate by % Minority for Cities in LA County

Preliminary Data Collected From LA County Website in Nov. 2004

% Hispanic or Non-White

110%

100%

90%

80%

70%

60%

50%

40%

30%

20%

10%

0%

Resid

ual R

ate

for

Pre

sid

ency,

Nov.,

2004

5.0%

4.5%

4.0%

3.5%

3.0%

2.5%

2.0%

1.5%

1.0%

.5%

0.0%

WhittierWestlake Village

West Hollywood

West Covina

WalnutTorrance

Temple City

South Pasadena

South Gate

South El Monte

Signal Hill

Sierra Madre

Santa Monica

Santa Fe Springs

Santa Clarita

San Marino

San Gabriel

San Fernando

San Dimas

Rosemead

Rolling Hills Estate

Rolling Hills

Redondo Beach

Rancho Palos Verdes

Pomona

Pico Rivera

Pasadena

Parmount

Palos Verdes Estates

Palmdale

Norwalk

Monterey Park

Montebello

Morovia

Maywood

Manhattan BeachMalibu

Lynwood

Los Angeles

Long Beach

Lomita

LawndaleLa Verne

La Puente

Lancaster

La Mirad

Lakewood

La Habra Heights

La Canada/Flintridge

Irwindale

Inglewood

Huntington Park

Hidden Hills

Hermosa Beach

Hawthorne

Hawaiian Gardens

GlendoraGlendale

Gardena

El Segundo

El MonteDuarate

Downey

Diamond Bar

Culver City

Cudahy

Covina

Compton

Commerce

Claremont

CerritosCarson

Calabasas

BurbankBeverly Hills

Bell Gardens

Bell Flower

Bell

Baldwin ParkAzusa

Avalon

Artesia

Arcadia

Alhambra

Agoura Hills

Pres. Residual Rate by Percent HS Graduation

for Cities in LA County in 2004

Per Cent High School Grad

110%

100%

90%

80%

70%

60%

50%

40%

30%

20%

Resid

ual R

ate

for

Pre

sid

ency,

Nov.

2004

5.0%

4.5%

4.0%

3.5%

3.0%

2.5%

2.0%

1.5%

1.0%

.5%

0.0%

City Size

gt 1000K

200-1000K

100-200K

50-100K

30-50K

20-30K

10-20K

5-10K

lt 5000

Total Population

WhittierWestlake Village

West Hollywood

West Covina

WalnutTorrance

Temple City

South Pasadena

South Gate

South El Monte

Signal Hill

Sierra Madre

Santa Monica

Santa Fe Springs

Santa Clarita

San Marino

San Gabriel

San Fernando

San Dimas

Rosemead

Rolling Hills Estate

Rolling Hills

Redondo Beach

Rancho Palos Verdes

Pomona

Pico Rivera

Pasadena

Parmount

Palos Verdes Estates

Palmdale

Norwalk

Monterey Park

Montebello

Morovia

Maywood

Manhattan BeachMalibu

Lynwood

Los Angeles

Long Beach

Lomita

Lawndale La Verne

La Puente

Lancaster

La Mirad

Lakewood

La Habra Heights

La Canada/Flintridge

Irwindale

Inglewood

Huntington Park

Hidden Hills

Hermosa Beach

Hawthorne

Hawaiian Gardens

GlendoraGlendale

Gardena

El Segundo

El MonteDuarate

Downey

Diamond Bar

Culver City

Cudahy

Covina

Compton

Commerce

Claremont

CerritosCarson

Calabasas

BurbankBeverly Hills

Bell Gardens

Bell Flower

Bell

Baldwin Park Azusa

Avalon

Artesia

Arcadia

Alhambra

Agoura Hills

What Causes these Relationships?

• Consider Fresno’s “experiment” of going from Votomatic style punchcards in 1996 to optical scan with precinct count in 2000

• The people voting remained essentially the same, only the voting system changed.

• Hence, difference in residual rates must be due to voting system.

Residual Votes in Fresno County

with Votomatic Punch in 1996

Percent Minority Voting Age Population in Tract

100%90%80%70%60%50%40%30%20%10%0%

Perc

ent

Pre

sid

ential R

esid

ual V

ote

s

8%

7%

6%

5%

4%

3%

2%

1%

0%

Residual Votes in Fresno County

with Optical Scan Precinct in 2000

Percent Minority Voting Age Population in Tract

100%90%80%70%60%50%40%30%20%10%0%

Perc

ent

Pre

sid

ential R

esid

ual V

ote

s

8%

7%

6%

5%

4%

3%

2%

1%

0%

Comparing Los Angeles with Other Counties

• Compare residual vote rates with adjoining Counties.

• If counties are similar, then those with higher residual vote rates have voting systems that are performing badly.

• Are counties similar?

Table 2: Residual Vote Rates for Los Angeles and Adjacent Counties

County Residual Vote Rate November 2004

Residual Vote Rate November 2005

Kern 1.51% 0.51%

Los Angeles 2.02% 3.52%

Orange 1.74% 1.13%

San Bernardino 1.06% 0.61%

Ventura 0.93% 0.72%

Note: In November 2004 the residual vote rate is for the presidential race. In November 2005, the residual vote rate is the lowest among all eight propositions.Sources: California Secretary of State’s Web Page and Los Angeles Counties web page.

Maybe Los Angeles is Just Different?

• Again consider November 2004 data.

• Look at precincts at LA border and compare them with contiguous precincts in adjoining counties.

• People should be similar, but voting systems are different.

• If difference in residual votes, then due to voting systems.

Conclusions

• Some voting systems more accurate than others, • Residual vote rates can identify problems with voting

systems,• Better data, such as breaking out undervotes and

overvotes by absentee, early-voting, and polling place voting in each precinct, would be very useful for assessing accuracy,

• The state of California should, after adjusting for differences in voters across counties, regularly prepare a “Voting Performance Report Card” to identify systems with low residual vote rates and those with high rates,

• Accuracy should be given as much weight as security in considering the performance of voting systems.