us voting system conformity assessment system prepared for travis county texas may 27, 2009

TTEEMM CCoonnssuullttiinngg,, LLPP

© Copyright 2009 TEM Consulting, LP - All Rights ReservedPresentation To Travis County, TX - May 27, 2009 Rev 1 – 05/22/09 - HSB

US Voting SystemConformity Assessment System

Prepared forTravis County Texas

May 27, 2009



Conformity Assessment

Conformity assessment is a system designed to assure that products and services meet requirements.

Certification is a common component in a conformity assessment system.

Certification of a product is a means of providing assurance that it complies with specified standards and other normative documents.



System Certification

Several types of certification systems exist:

•Some comprise type testing only

•Other systems include initial testing and field surveillance

•Still others include initial testing of a product and assessment of its suppliers’ quality systems, followed by routine audits that take into account the factory quality system and the testing of samples from the factory and the open market.



Voting System Certification

In the US, Voting System Certification includes:

• National testing and certification by the EAC• State testing and certification• Local acceptance testing (recommended)• Assessment of its suppliers’ quality systems• field surveillance by both EAC and state officials



International Conformity Assessment

Conformity assessment is an international concern to assure the safety of food, pharmaceuticals and other critical items. It is also a necessity to support international trade.

To meet these needs a number of treaties and agreements have been created. These agreements are implemented by processes, created and supervised by organizations such as:

• International Standards Organization (ISO)• International Laboratory Assessment Cooperation (ILAC)



Purpose of Presentation

This presentation discusses the elements common to conformity assessment systems and how these are used in the US Election System for certification and decertification of voting equipment.



Components of a Certification System

•Initial Type Testing

•Assessment of the Supplier’s Quality System

•Field Information & Feedback

•User Involvement



Key Participants•EAC

•Voting System Test Laboratories

•Texas Secretary of State, Elections Division

•Travis County Office of Elections

•Vendors

•Voters



National Program State & Local Programs

The primary concern of the EAC program is that the system design, as represented in the system submitted for testing, meets the requirements

The primary concern of the state and local election official is that: •Meets the specific needs of a jurisdiction•the units delivered meet and continue to meet the requirements over their useful life.



Key Issues for Certification System

•What is a minimal acceptable system?

•Are the testing lab/testers/lab assessors qualified?

•Will the vendor deliver units within manufacturing tolerances to those tested?

•How will the election officials know if non-compliant units are delivered and what corrective actions can it take?

•Will election officials and poll workers use the systems as intended?



EAC Testing and Certification Program

VVSGSpecification

Archived Software

Field Experience

Election System

Conformity Assessment System

Fee

dbac

k

Trusted Archive, e.g. NIST NSRL

Vendor(Candidate

Voting System)

TGDC & NIST

State Certification

Acceptance & Pre-Election Testing

System Testing

System Testing

Verification

Verification

VSTL Evaluation

EAC Determination

Su

bm

iss

ion

Fee

dbac

k



Processes

Processes are necessary to implement the components of a conformity assessment system for voting systems and to provide reasonable assurance that the key concerns of such a system are carefully addressed.



Processes

• Technical Standard• Establish normative standard

2002 VSS & EAC VVSG• Refine and Keep standard current

• Laboratory Accreditation• NIST NVLAP Accredit to ISO 17025 to assure:

• Technical competence• Management and quality process control

• EAC Accredit as Voting System Test Laboratory



Processes

• Technical Reviewers• Review and give recommendations to EAC• TR’s will have specialized qualifications for various topics• EAC will perform routine performance evaluations

• Product Evaluation• Test plan• Testing and test report• Witness testing will be part of the process• Test report review and system certification



Processes

• Vendor Registration• Review the vendor configuration control system• Review the vendor quality control system



EAC / NISTTGDC

Technical Guidelines

EAC / NISTTGDC

Technical GuidelinesCandidate System

Accredited

Test Laboratory

Accredited

Test Laboratory EACEAC Technical Review

Team

Technical Review

Team

Certified SystemSystem

Description

System

DescriptionNIST

Software Ref. Lib.

NIST

Software Ref. Lib.

Vendor

Quality & ConfigurationControl Processes

Vendor

Quality & ConfigurationControl Processes

State & LocalEvaluation

State & LocalEvaluation

Field Deploymentand Use

Field Deploymentand Use

Receiving InspectionReceiving Inspection



EAC / NISTTGDC


EAC / NISTTGDC


Accredited

Test Laboratory

Accredited

Test Laboratory

EACEACTechnical Review

Team

Technical Review

Team

NIST

NVLAP&

Software Ref. Lib.

NIST

NVLAP&

Software Ref. Lib.

VendorsVendors

State & LocalElection Officials

State & LocalElection Officials

CitizensCitizensLocal

Receiving Inspectors

Local

Receiving Inspectors



TGDC&

NIST

VSTL

VVSGSpecification

Build Executable Code

Submission

Source Code

Executable Code

Build Environment

Election Software

Certification, Delivery & Verification Process

System Testing

Trusted Archive, e.g. NIST NSRLDeposit

Manufacturer(Source Code, Build Environment, Voting

System)

Source Code Review

State Certification

Acceptance & Pre-Election Testing

System Testing

System Testing

Verific

ation

Verification



Conclusion

A well constructed certification system provides satisfactory answers to central issues:

•What is a minimal acceptable system?

•Are the testing lab/testers/lab assessors qualified?

•Will the vendor deliver units within manufacturing tolerances to those tested?

•Will the election officials know if non-compliant units are delivered and what actions can they take?

•Will election officials and poll workers use the systems as intended?

us voting system conformity assessment system prepared for travis county texas may 27, 2009

Documents