Smart Columbus
Connected Vehicle Environment (CVE) Test Plan
for the Smart Columbus
Demonstration Program
DRAFT REPORT | November 26, 2019
Produced by City of Columbus
Notice
This document is disseminated under the sponsorship of the Department of Transportation in the interest of information exchange. The United States Government assumes no liability for its contents or use thereof.
The U.S. Government is not endorsing any manufacturers, products, or services cited herein and any trade name that may appear in the work has been included only because it is essential to the contents of the work.
Acknowledgement of Support
This material is based upon work supported by the U.S. Department of Transportation under Agreement No. DTFH6116H00013.
Disclaimer
Any opinions, findings, and conclusions or recommendations expressed in this publication are those of the Author(s) and do not necessarily reflect the view of the U.S. Department of Transportation.
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | i
Abstract
The purpose of the Smart Columbus Master Test Plan is to establish a common framework for conducting
Connected Vehicle Environment project testing activities. The plan facilitates processes among project
stakeholders including the City of Columbus; suppliers of hardware components including roadside units,
onboard units, message handling equipment, signal controllers; application suppliers; and other parties.
The plan categorizes all components that make up the system of interest; outlines the testing strategy;
defines the test tasks; defines interactions with other system elements; and provides governance for
executing all testing activities and related equipment including tools for logging, tracking, monitoring, and
reporting test outcomes.
The master test plan has the following primary goals:
1. Evaluate how a system of interest conforms to allocated requirements
2. Assess whether the system of interest satisfies its intended use and user needs
These determinations include a blend of analysis, demonstration, inspection and testing of various products,
systems, and data to provide final acceptance of the system and move to the next project phase.
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | iii
Table of Contents
3.1. Test Components ........................................................................................................... 7
3.2. Functionality to be Tested ............................................................................................. 8
3.2.1. Roadside Equipment........................................................................................................ 8
3.2.2. Light-Duty Vehicle Onboard Unit ...................................................................................... 9
3.2.3. Heavy-Duty Vehicle Onboard Unit.................................................................................. 10
3.2.4. Emergency Vehicle Onboard Unit .................................................................................. 11
3.2.5. Transit Vehicle Onboard Unit.......................................................................................... 12
3.2.6. Traffic Connected Vehicle Management System ............................................................ 14
3.2.7. Transit Connected Vehicle Management System ........................................................... 15
3.3. Functionality not to be tested ..................................................................................... 15
3.3.1. Heavy-Duty Vehicle Onboard Unit.................................................................................. 15
3.3.2. Data Distribution to Third Parties .................................................................................... 16
4.1. Testing Approach ......................................................................................................... 17
4.2. Testing Types ................................................................................................................ 17
4.2.1. First Article Testing ......................................................................................................... 18
4.2.2. Failure Reporting............................................................................................................ 18
4.2.3. Post-Installation Testing.................................................................................................. 19
4.2.4. Final System Testing ...................................................................................................... 19
4.2.5. Operational Period ......................................................................................................... 19
4.3. Testing Schedule .......................................................................................................... 19
4.4. Roles .............................................................................................................................. 20
4.5. Test Tools ...................................................................................................................... 22
4.5.1. Connected Vehicle Test Tool .......................................................................................... 22
4.5.2. Onboard Unit ................................................................................................................. 22
4.5.3. Test Result Log .............................................................................................................. 22
4.5.4. Defect Tracker ................................................................................................................ 23
4.5.5. Change Request Log ..................................................................................................... 23
4.6. Environmental Needs................................................................................................... 23
4.6.1. Bench Facility ................................................................................................................. 23
4.6.2. Demonstration Area Facility ........................................................................................... 24
4.6.3. Test Preparation ............................................................................................................. 25
Table of Contents
iv | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
4.7. Measures and Metrics .................................................................................................. 26
4.8. Test Criteria ................................................................................................................... 26
4.8.1. Item Pass/Fail ................................................................................................................ 26
4.8.2. Testing Suspension and Resumption ............................................................................. 27
6.1. Onboard Unit ................................................................................................................ 31
6.2. Onboard Unit Applciation ............................................................................................ 36
6.3. Roadside Unit ............................................................................................................... 38
6.4. Message Handler .......................................................................................................... 42
6.5. Traffic Signal controller................................................................................................ 46
6.6. Traffic CV Management System .................................................................................. 48
6.7. Transit CV Management System ................................................................................. 55
8.1. Onboard Unit ................................................................................................................ 65
8.2. Onboard Unit Applciation ............................................................................................ 78
8.3. Roadside Unit ............................................................................................................... 83
8.4. Message Handler .......................................................................................................... 88
8.5. Traffic Signal controller................................................................................................ 96
8.6. Traffic CV Management System ................................................................................ 101
8.7. Transit CV Management System ............................................................................... 115
Table of Contents
Connected Vehicle Environment Test Plan – Draft Report | Smart Columbus Program | v
List of Tables
Table 1: Connected Vehicle Environment Project Scope .......................................................................... 2
Table 2: Device/Subsystem Test Matrix .................................................................................................... 7
Table 3: Test Phase and Timeframes ..................................................................................................... 19
Table 4: Test Roles and Responsibilities ................................................................................................ 20
Table 5: Onboard Unit (Test Cases) ....................................................................................................... 31
Table 6: Onboard Unit Application Test Cases ........................................................................................ 36
Table 7: Roadside Unit Test Cases......................................................................................................... 38
Table 8: Message Handler Test Cases ................................................................................................... 42
Table 9: Traffic Signal Controller Test Cases .......................................................................................... 46
Table 10: Traffic CV Management System Test Cases ........................................................................... 48
Table 11: Transit CV Management System Test Cases ........................................................................... 55
Table 12: Testing Scenarios with Requirements Traceability ................................................................... 57
Table 13: Onboard Unit Test Procedures ................................................................................................ 65
Table 14: Onboard Unit Application Test Procedures .............................................................................. 78
Table 15: Roadside Unit Test Procedures ............................................................................................... 83
Table 16: Message Handler Test Procedures ......................................................................................... 88
Table 17: Traffic Signal Controller Test Procedures ................................................................................ 96
Table 18: Traffic CV Management System Test Procedures.................................................................. 101
Table 19: Transit CV Management System Test Procedures................................................................. 115
Table 20: Test Result Log ..................................................................................................................... 117
Table 21: Defect Tracker ...................................................................................................................... 118
Table 22: Change Request Log ............................................................................................................ 118
Table 23: Test Summary Report ........................................................................................................... 118
Table 24: Defect Matrix ........................................................................................................................ 119
Table 25: Test Case Status .................................................................................................................. 119
Table 26: Numbering Convention ......................................................................................................... 121
Table 27: Acronym List ......................................................................................................................... 123
Table 28: Glossary ............................................................................................................................... 125
List of Figures
Figure 1: Numbering Convention ......................................................................................................... 121
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 1
Chapter 1. Project Background
In 2016, the U.S. Department of Transportation (USDOT) awarded $40 million to the City of Columbus,
Ohio, as the winner of the Smart City Challenge. With this funding, Columbus intends to address the most
pressing community-centric transportation problems by integrating an ecosystem of advanced and
innovative technologies, applications, and services to bridge the sociotechnical gap and meet the needs of
residents of all ages and abilities. In conjunction with the Smart City Challenge, Columbus was also
awarded a $10 million grant from Paul G. Allen Philanthropies to accelerate the transition to an electrified,
low-emissions transportation system.
With the award, the City established a strategic Smart Columbus program with the following vision and
mission:
Smart Columbus Vision: Empower residents to live their best lives through responsive, innovative,
and safe mobility solutions
Smart Columbus Mission: Demonstrate how Intelligent Transportation Systems (ITS) and equitable
access to transportation can have positive impacts on every day challenges faced by cities
To enable these new capabilities, the Smart Columbus program is organized into three focus areas
addressing unique user needs: enabling technologies, emerging technologies, and enhanced human
services. The Connected Vehicle (CV) Environment (CVE) primarily addresses needs in the enabling
technologies focus area. The CVE project is one of the eight projects in the Smart Columbus program and is
a significant enabler to other technologies delivered through the other seven projects. The CVE project will
integrate smart traveler applications, connected vehicles, and automated vehicles into its transportation
network by focusing on deploying CV infrastructure and CV applications.
CV Infrastructure: The project will focus on building out the physical and logical CV infrastructure,
which will consist of CV hardware and software (e.g. roadside units (RSUs), onboard units (OBUs),
front and backhaul communications, equipment interfaces, etc.). The CVE will generate the needed
transportation-related data that the applications use.
CV Applications and Data: The project scope also consists of deploying CV-specific applications
that will leverage the data generated by the infrastructure to deliver real-time safety and mobility
services. Data will be collected, related, stored, and made available for use in other Smart Columbus
project applications.
The CVE is expected to enhance safety and mobility for vehicle operators and improve pedestrian safety in
School Zones by deploying CV infrastructure on the roadside and CV equipment in vehicles. The CVE will
also provide sources of high-quality data for traffic management and safety purposes.
The foundation for the CVE is the Columbus Traffic Signal System (CTSS), which is a high-speed network
backbone. When complete, the CTSS will interconnect 1,250 traffic signals in the Columbus region and
provide uniform signal coordination capability throughout the system. CTSS Phase D, which will connect all
CVE corridors except for Alum Creek Drive, is expected to be complete by Q3 2019. An expansion of the
CTSS to connect Alum Creek Drive will be included in the CVE project.
The CV infrastructure deployment will occur along four major corridors/areas. The deployment of in-vehicle
devices will target populations that are located near, or frequently use, the corridors where infrastructure is
deployed. Table 1 lists the improvements associated with the CVE.
Chapter 1. Project Background
2 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
Table 1: Connected Vehicle Environment Project Scope
Infrastructure Applications and Data
100+ RSUs
The project will install RSUs and necessary communications equipment at ~90 signalized intersections in the project areas.
1,500 – 1,800 OBUs
The project will install onboard units (OBUs) on participating private, fleet, emergency, transit, and freight vehicles.
CV Applications
The project will deploy vehicle-to-vehicle (V2V) safety, vehicle-to-infrastructure (V2I) safety, and V2I mobility applications.
Data Capture
The project will capture, relate, store, and respond to data generated by the infrastructure, used by the applications for traffic management.
Source: City of Columbus
The intent of the CVE project is to improve safety and mobility of travelers by deploying CV technology as
part of a larger initiative within the City to improve the overall transportation system. CV technology will also
be deployed to support the improvement in freight operations, another of the City’s goals.
Collectively, CV is just one component, but if it proves to be effective, other projects can also benefit from
the positive outcomes. Recall that the goal of the CVE is not to develop applications to a high level of
maturity. It is to leverage what has already been developed. Thus, it is important for the reader to
understand that the ability of the CVE to address the user needs captured in the ConOps depends on the
availability of hardware and software solutions that have been previously deployed (and subsequently
improved upon). To this end, throughout the CVE systems engineering process, several applications have
been considered – each application was scrutinized in detail to ensure that only applications that were ready
for deployment are included in the deployment of the CVE. Performance requirements detailed in the
System Requirements document (see V2V Safety, V2I Safety, and V2I Mobility functional groups), outline
expectations for each application that is deployed. The implementation of software is expected to
demonstrate efficacy of the deployed infrastructure.
The applications and technology to be deployed as part of the CVE are the same (or very similar) to
applications and technology employed in other connected vehicle projects. As similar applications are
developed and employed as part of the CV Pilot projects, their maturity will continue to increase. It is
expected that prospective vendors are perpetually making improvements to applications based on
experience in testing and implementation. Thus, the design and implementation of the CVE will draw on
improvements made to applications through these development efforts.
The Architecture Reference for Cooperative and Intelligent Transportation (ARC-IT)1 and its predecessor,
the CV Reference Implementation Architecture (CVRIA),2 are resources that provide descriptions of CV
applications that have been researched in the context of the National ITS architecture. Furthermore, the
Open Source Application Development Portal (OSADP) contains software for applications that have been
developed.3 When possible, applications on ARC-IT, CVRIA and OSADP will be used as-is or will have
minimal modifications made to address user needs documented in the ConOps.
Given that the primary scope of the CVE is to realize the benefits of deploying CV technology into an
operational environment, only applications that have demonstrated sufficient levels of development and
testing are being considered for implementation. It is expected that prospective vendors are perpetually
1 https://local.iteris.com/arc-it/ 2 https://local.iteris.com/cvria/ 3 Open Source Application Development Portal. https://www.itsforge.net/
Chapter 1. Project Background
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 3
making improvements to applications based on experience in testing and implementation. However, the
CVE will be designed in such a way that added functionality concepts (not implemented as part of this
project, that require further development) can be integrated with the CVE once development and testing
have matured to a point where applications are deployment-ready. Additionally, due to the networked nature
of devices in the CVE, several policies and constraints related to information technology (IT) and data
security are expected to be developed as part of the deployment. The backhaul network which supports
roadside equipment in the CVE will require the establishment of a new network (on existing dark fiber). City
of Columbus Department of Technology (responsible for managing Columbus’ fiber network) has been
engaged to establish necessary network security policies and design.
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 5
Chapter 2. Risks and Contingencies
Some systems, functionality, or devices may not be available during initial testing activities. Listed below are
a few risks and contingencies for initial CVE Testing.
Risk: IPv6 is not available during initial testing
Contingency: Onboard Unit (OBU) Testing to the State of Ohio Security Credential Management
System (SCMS) and Firmware Over-the-Air updates will not be able to be tested. OBU IPv6 testing
will occur once the IPv6 system is available.
Risk: The Smart Columbus Operating System (Operating System) may not be available/accessible
during initial testing.
Contingency: A laptop computer connected to and simulating the Operating System. Data intended
to be sent to the Operating System will be sent to and stored on this laptop, demonstrating the
process of sending data to the Operating System.
Risk: A Message Handler hardware may not be available during initial testing
Contingency: A laptop computer connected to the Signal Controller and Roadside Unit (RSU) will be
set up to perform the Message Handler functions.
Risk: Some OBU applications may not be available during initial testing
Contingency: OBU applications will be tested in the controlled environment as they are available,
prior to being released to OBUs installed in vehicles. Applications will be installed in OBUs installed in
vehicles through an Over-the-Air Firmware update.
Additional testing activities (phases) will be required if a system, functionality, or device is not available during initial testing. These additional testing actives will be coordinated and scheduled based on all stakeholder availability.
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 7
Chapter 3. Test Items
3.1. TEST COMPONENTS
Table 2 lists the CVE devices and subsystems to be tested, and the devices and subsystems against which
each will be tested.
Table 2: Device/Subsystem Test Matrix
Device/Subsystem Under Test Devices/Subsystems Tested Against
Kapsch RSU Message Handler
Siemens OBU
SCMS
CVE Network
Danlaw RSU Message Handler
Siemens OBU
SCMS
CVE Network
Siemens RSU Message Handler
Siemens OBU
SCMS
CVE Network
Message Handler Kapsch RSU
Danlaw RSU
Siemens RSU
Econolite Signal Controller
Siemens Signal Controller
Traffic CV Management System (TCVMS)
CVE Network
Chapter 3. Test Items
8 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
Device/Subsystem Under Test Devices/Subsystems Tested Against
Siemens OBU Kapsch RSU
Danlaw RSU
Siemens RSU
Transit CV Management System (TrCVMS) (transit vehicles)
SCMS
Siemens OBU Firmware server
CVE Network
Forward Collision Warning (FCW) light-duty vehicle (LDV) application
Electronic Emergency Brake Light (EEBL) LDV application
Lane Change Warning (LCW) LDV application
Intersection Movement Assist (IMA) LDV application
Red Light Violation Warning (RLVW) LDV application
Reduce Speed School Zone Waring (RSSZ) LDV application
Signal Priority for transit and heavy-duty vehicles (HDVs)
Signal Pre-Emption for emergency vehicles (EVs)
Traffic CV Management System (TCVMS)
Message Handler
Operating System
CVE Network
City CVE IT Network RSUs
Message Handler
SCMS
Siemens OBU Firmware server
Source: City of Columbus
3.2. FUNCTIONALITY TO BE TESTED
This section contains a high-level description of the Smart Columbus CVE features that will be tested as
part of this Test Plan.
3.2.1. Roadside Equipment
The following functionality of the Smart Columbus CVE Roadside Equipment will be tested:
RSU’s ability to broadcast J2735 Signal Phase and Timing (SPaT) and MAP messages
The successful testing of this functionality indicates:
The Signal Controller sends SPaT data to the Message Handler
The Message Handler encodes SPaT data into J2735 SPaT messages
The Message Handler sends J2735 SPaT messages and MAP messages to the RSU
RSU’s ability to broadcast Radio Technical Commission for Maritime Services (RTCM) messages
The successful testing of this functionality indicates:
The CVE Network supports connection between the Position Correction System and the
Message Handler
Chapter 3. Test Items
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 9
The Position Correction System sends RTCM data to the Message Handler
The Message Handler encodes RTCM data into J2735 RTCM messages
The Message Handler sends J2735 RTCM messages to the RSU
RSU’s ability to broadcast Signal Status Message (SSM)
The successful testing of this functionality indicates:
The Signal Controller sends signal priority status data to the Message Handler
The Message Handler encodes signal priority status data into J2735 SSMs
The Message Handler sends J2735 SSMs to the RSU
RSU’s ability to request and receive IEEE 1609.2 certificates from the ODOT SCMS
The successful testing of this functionality indicates:
The RSU determines it needs additional certificates
The RSU sends a certificate request to the ODOT SCMS
RSU’s ability to forward Basic Safety Messages (BSMs) and SRMs to the Message Handler
The successful testing of this functionality indicates:
The RSU is configured properly to forward messages to the Message Handler
RSU’s ability to broadcast RSSZ when the School Zone sign is flashing/active.
The successful testing of this functionality indicates:
The CVE Network supports connection between the School Zone System and the Message
Handler
The School Zone System sends School Zone sign data to the Message Handler
The Message Handler determines when the School Zone sign is flashing/active
The Message Handler encodes a J2735 RSSZ message when the School Zone sign is
flashing/active
The Message Handler sends J2735 RSSZ messages to the RSU while the School Zone sign
is flashing
3.2.2. Light-Duty Vehicle Onboard Unit
The following functionality of the Smart Columbus CVE Light-Duty (LDV) OBU will be tested:
LDV OBU’s ability to broadcast Part I BSMs
LDV OBU’s ability to receive and process RTCM
The successful testing of this functionality indicates:
The CVE Network supports connection between the Position Correction System and the
Message Handler
The Position Correction System sends RTCM data to the Message Handler
The Message Handler encodes RTCM data into J2735 RTCM messages
The Message Handler sends J2735 RTCM messages to the RSU
The RSU broadcasts J2735 RTCM messages
LDV OBU updates position based on RTCM
LDV OBU’s ability to provide EEBL, FCW, LCW, and IMA warnings to drivers
The successful testing of this functionality indicates the LDV OBU’s EEBL, FCW, LCW, and IMA
OBU applications are functioning as intended
Chapter 3. Test Items
10 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
LDV OBU’s ability to provide RLVW warnings to drivers
The successful testing of this functionality indicates:
The Signal Controller sends SPaT data to the Message Handler
The Message Handler encodes SPaT data into J2735 SPaT messages
The Message Handler sends J2735 SPaT messages and MAP messages to the RSU
The RSU broadcasts SPaT and MAP messages
The LDV OBU RLVW application is functioning as intended
LDV OBU’s ability to provide RSSZ warnings to drivers
The successful testing of this functionality indicates:
The CVE Network supports connection between the School Zone System and the Message
Handler
The School Zone System sends School Zone sign data to the Message Handler
The Message Handler determines when the School Zone sign is flashing/active
The Message Handler encodes a J2735 RSSZ message when the School Zone sign is
flashing/active
The Message Handler sends J2735 RSSZ messages to the RSU while the School Zone sign
is flashing/active
The RSU broadcasts RSSZ messages
OBU RSSZ application is functioning as intended
LDV OBU’s ability to request and receive IEEE 1609.2 certificates from the ODOT SCMS through an
RSU.
The successful testing of this functionality indicates:
The RSU is broadcasting Wireless Access in Vehicular Environments (WAVE) Service
Advertisements (WSA) containing WAVE Routing Advertisements (WRA) for IPv6 Services
The LDV OBU receives and process WSAs containing WRAs
The LDV OBU determines it needs additional certificates
The LDV OBU sends a certificate request to the ODOT SCMS
The RSU acts as an IPv6 Gateway
The CVE Network supports IPv6 connectivity to the ODOT SCMS
3.2.3. Heavy-Duty Vehicle Onboard Unit
The following functionality of the Smart Columbus CVE HDV OBU will be tested:
HDV OBU’s ability to broadcast Part I BSMs
HDV OBU’s ability to receive and process RTCM
The successful testing of this functionality indicates:
The CVE Network supports connection between the Position Correction System and the
Message Handler
The Position Correction System sends RTCM data to the Message Handler
The Message Handler encodes RTCM data into J2735 RTCM messages
The Message Handler sends J2735 RTCM messages to the RSU
The RSU broadcasts J2735 RTCM messages
HDV OBU updates position based on RTCM
Chapter 3. Test Items
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 11
HDV OBU’s ability to generate and broadcast a J2735 SRM to request Signal Priority
The successful testing of this functionality indicates:
The Signal Controller is sending SPaT data to the Message Handler
The Message Handler is encoding SPaT data into J2735 SPaT messages
The Message Handler is sending J2735 SPaT messages and MAP messages to the RSU
The RSU is broadcasting WSAs, advertising Signal Priority/Preemption, SPaT, and MAP
messages
The HDV OBU Signal Priority Request application is receiving and processing WSAs, SPaT,
and MAP messages
HDV OBU’s ability to receive and logs a J2735 Signal Status Messages (SSM)
The successful testing of this functionality indicates:
The HDV OBU is broadcasting J2735 SRMs
The RSU is forwarding SRMs to the Message Handler
The Message Handler is sending signal priority requests to the Signal Controller
The Signal Controller is processing signal priority requests
The Signal Controller is sending signal priority status data to the Message Handler
The Message Handler is encoding signal priority status data into J2735 SSMs
The Message Handler is sending J2735 SSMs to the RSU
The RSU is broadcasting SSMs
The HDV Signal Priority Request application is functioning as intended
HDV OBU’s ability to request and receive IEEE 1609.2 certificates from the Ohio Department of
Transportation (ODOT) SCMS through an RSU.
The successful testing of this functionality indicates:
The RSU is broadcasting WSAs containing WRAs for IPv6 Services
The HDV OBU receives and process WSAs containing WRAs
The HDV OBU determines it needs additional certificates
The HDV OBU sends a certificate request to the ODOT SCMS
The RSU acts as an IPv6 Gateway
The CVE Network supports IPv6 connectivity to the ODOT SCMS
3.2.4. Emergency Vehicle Onboard Unit
The following functionality of the Smart Columbus CVE EV OBU will be tested:
EV OBU’s ability to broadcast Part I BSMs
EV OBU’s ability to receive and process RTCM
The successful testing of this functionality indicates:
The CVE Network supports connection between the Position Correction System and the
Message Handler
The Position Correction System sends RTCM data to the Message Handler
The Message Handler encodes RTCM data into J2735 RTCM messages
The Message Handler sends J2735 RTCM messages to the RSU
The RSU broadcasts J2735 RTCM messages
Chapter 3. Test Items
12 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
EV OBU updates position based on RTCM
EV OBU’s ability to generate and broadcast an SRM
The successful testing of this functionality indicates:
The Signal Controller is sending SPaT data to the Message Handler
The Message Handler is encoding SPaT data into J2735 SPaT messages
The Message Handler is sending J2735 SPaT messages and MAP messages to the RSU
The RSU is broadcasting WSA’s, advertising Signal Priority/Preemption, SPaT, and MAP
messages
The EV OBU Signal Preemption Request application is receiving and processing WSAs, SPaT,
and MAP messages
EV OBU’s ability to receive and logs an SSM
The successful testing of this functionality indicates:
The EV OBU is broadcasting J2735 SRMs
The RSU is forwarding SRMs to the Message Handler
The Message Handler is sending signal preemption requests to the Signal Controller
The Signal Controller is processing signal preemption requests
The Signal Controller is sending signal preemption status data to the Message Handler
The Message Handler is encoding signal preemption status data into J2735 SSMs
The Message Handler is sending J2735 SSMs to the RSU
The RSU is broadcasting SSMs
The EV Signal Preemption Request application is functioning as intended
EV OBU’s ability to request and receive IEEE 1609.2 certificates from the ODOT SCMS through an
RSU.
The successful testing of this functionality indicates:
The RSU is broadcasting WSAs containing WRAs for IPv6 Services
The EV OBU receives and process WSAs containing WRAs
The EV OBU determines it needs additional certificates
The EV OBU sends a certificate request to the ODOT SCMS
The RSU acts as an IPv6 Gateway
The CVE Network supports IPv6 connectivity to the ODOT SCMS
3.2.5. Transit Vehicle Onboard Unit
The following functionality of the Smart Columbus CVE Transit Vehicle OBU will be tested:
Transit OBU’s ability to broadcast Part I BSMs
Transit OBU’s ability to receive and process RTCM
The successful testing of this functionality indicates:
The CVE Network supports connection between the Position Correction System and the
Message Handler
The Position Correction System sends RTCM data to the Message Handler
The Message Handler encodes RTCM data into J2735 RTCM messages
The Message Handler sends J2735 RTCM messages to the RSU
Chapter 3. Test Items
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 13
The RSU broadcasts J2735 RTCM messages
Transit OBU updates position based on RTCM
Transit OBU’s ability to generate and broadcast an SRM
The successful testing of this functionality indicates:
The Signal Controller is sending SPaT data to the Message Handler
The Message Handler is encoding SPaT data into J2735 SPaT messages
The Message Handler is sending J2735 SPaT messages and MAP messages to the RSU
The RSU is broadcasting WSA’s, advertising Signal Priority/Preemption, SPaT, and MAP
messages
The Transit OBU Signal Priority Request application is receiving and processing WSAs, SPaT,
and MAP messages
Transit OBU’s ability to log EEBL, FCW, LCW, and IMA events
The successful testing of this functionality indicates:
The Transit OBU’s EEBL, FCW, LCW, and IMA applications are functioning as intended
The Transit OBU message logging process is functioning as intended
Transit OBU’s ability to log RLVW events
The successful testing of this functionality indicates:
The Signal Controller sends SPaT data to the Message Handler
The Message Handler encodes SPaT data into J2735 SPaT messages
The Message Handler sends J2735 SPaT messages and MAP messages to the RSU
The RSU broadcasts SPaT and MAP messages
The Transit OBU’s RLVW application is functioning as intended
The Transit OBU message logging process is functioning as intended
Transit OBU’s ability to log RSSZ events
The successful testing of this functionality indicates:
The CVE Network supports connection between the School Zone System and the Message
Handler
The School Zone System sends School Zone sign data to the Message Handler
The Message Handler determines when the School Zone sign is flashing/active
The Message Handler encodes a J2735 RSSZ message when the School Zone sign is
flashing/active
The Message Handler sends J2735 RSSZ messages to the RSU while the School Zone sign
is flashing/active
The RSU broadcasts RSSZ messages
The Transit OBU’s RSSZ application is functioning as intended
The Transit OBU message logging process is functioning as intended
Transit OBU’s ability to receive and logs an SSM
The successful testing of this functionality indicates:
The Transit OBU is broadcasting J2735 SRMs
The RSU is forwarding SRMs to the Message Handler
The Message Handler is sending signal priority requests to the Signal Controller
The Signal Controller is processing signal priority requests
Chapter 3. Test Items
14 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
The Signal Controller is sending signal priority status data to the Message Handler
The Message Handler is encoding signal priority status data into J2735 SSMs
The Message Handler is sending J2735 SSMs to the RSU
The RSU is broadcasting SSMs
The Transit OBU Signal Priority Request application is functioning as intended
Transit OBU’s ability to send log files to the TrCVMS
The successful testing of this functionality indicates:
The Transit OBU logs events
The Transit OBU is connected to the Transit vehicles internal communication network
The Transit OBU determines when the Transit vehicle is in communication range of the Transit
vehicle garage network
The TrCVMS system is connected to Transit vehicle garage communication network
Transit OBU’s ability to request and receive IEEE 1609.2 certificates from the ODOT SCMS through
an RSU.
The successful testing of this functionality indicates:
The RSU is broadcasting WSAs containing WRAs for IPv6 Services
The Transit OBU receives and process WSAs containing WRAs
The Transit OBU determines it needs additional certificates
The Transit OBU sends a certificate request to the ODOT SCMS
The RSU acts as an IPv6 Gateway
The CVE Network supports IPv6 connectivity to the ODOT SCMS
3.2.6. Traffic Connected Vehicle Management System
The following functionality of the Smart Columbus CVE TCVMS will be tested:
Traffic Management Staff’s ability to send MAP message to Message Handler at each applicable
roadside location using the TCVMS
The successful testing of this functionality indicates:
The MAP generation tool User Interface is functioning as intended
The MAP generation process is functioning as intended
The CVE supports connectivity between the TCVMS and the Message Handler at each
applicable Roadside location
Traffic Management Staff’s ability to send RSSZ message to Message Handler at each applicable
roadside location using the TCVMS
The successful testing of this functionality indicates:
The RSSZ generation tool User Interface is functioning as intended
The RSSZ generation process is functioning as intended
The CVE supports connectivity between the TCVMS and the Message Handler at each
applicable Roadside location
TCVMS’ ability to log SPaT, RTCM, SSM, SRM, BSM received from Message Handlers at each
applicable Roadside location
The successful testing of this functionality indicates:
Chapter 3. Test Items
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 15
The CVE supports connectivity between the TCVMS and the Message Handler at each
applicable Roadside location
RSUs forward BSMs and SSMs to the Message Handler
Signal Controllers send SPaT data and signal priority/preemption data to Message Handlers
Message Handlers encode SPaT and SSM messages
Message Handler send SPaT, RTCM, SSM, SRM, and BSM messages to the TCVMS
The TCVMS logging process is functioning as intended
TCVMS’ ability to send SPaT, MAP, RSSZ, RTCM, SSM, SRM, and BSM to the Operating System
The successful testing of this functionality indicates:
The CVE supports connectivity between the TCVMS and the Operating System
The TCVMS logging process is functioning as intended
The TCVMS distribution process is functioning as intended
TCVMS’ ability to manage RSUs
The successful testing of this functionality indicates:
The CVE supports connectivity between the TCVMS and the Message Handler at each
applicable roadside location
The TCVMS sends RSU Command and Control messages to Message Handlers at each
applicable Roadside location
Message Handlers send Command and Control messages to RSUs
RSUs send Command and Control responses to Message Handlers
Message Handlers send RSU Command and Control responses to the TCVMS
3.2.7. Transit Connected Vehicle Management System
The following functionality of the Smart Columbus CVE TrCVMS will be tested:
TrCVMS’ ability to receive and archive log files from transit vehicle OBUs
The successful testing of this functionality indicates:
The TrCVMS system is connected to the transit vehicle garage communication network
Transit OBUs log events
Transit OBUs are connected to the transit vehicle internal communication network
Transit OBUs determine when transit vehicles are in communication range of the transit
vehicle garage network
Transit vehicles send log files to the TrCVMS
TrCVMS decodes and stores data contained in transit vehicle OBU log files
3.3. FUNCTIONALITY NOT TO BE TESTED
This section contains a high-level description of the Smart Columbus CVE that will not be tested as part of
this Test Plan.
3.3.1. Heavy-Duty Vehicle Onboard Unit
The following functionality of the Smart Columbus CVE HDV OBU will not be tested:
OBU’s ability to broadcast BSM Part II
Chapter 3. Test Items
16 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
3.3.2. Data Distribution to Third Parties
The Operating System’s distribution of CV data will not be tested as part of this Test Plan.
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 17
Chapter 4. Approach
4.1. TESTING APPROACH
The following five methods will be utilized to verify the operation and functionality of individual components,
subsystems and the overall Smart Columbus CVE:
1. Inspection: A nondestructive examination of the device/system using one or more of the senses
(visual, auditory, smell, tactile). This may include simple physical handling and measurements.
2. Demonstration: Exercising the device/system as it is intended to operate/function to verify results are
as planned or expected.
3. Test: Verification of the device/system using a controlled and predefined series of inputs, data, or stimuli
to ensure the system will produce a very specific and predefined output as specified by the
requirements.
4. Analysis: Verification of the device/system using models, calculations and testing equipment. Analysis
allows someone to make predictive statements about the typical performance of the device, application,
or system as a whole based on the confirmed test results of a sample set or by combining the outcome
of individual tests to conclude something new about the device, application, or system. It is often used
to predict the breaking point or failure of a device, application, or system by using nondestructive tests
to extrapolate the failure point.
5. Documentation: Review of documentation including, but not limited to, Test Reports from independent
labs/facilities.
4.2. TESTING TYPES
Each CVE device needs to be configured and tested prior to deployment to ensure proper operation within
the system. Configurations need to be tested prior to deployment, as once deployed, access to the
equipment will be limited; a bucket truck will be required in the event RSUs cannot be accessed, and OBUs
will not be accessible after they are installed in a vehicle.
Device functionality and component integration need to be verified before the system is deployed
as well as after equipment is installed at roadside locations and in vehicles. Testing will be completed
in the following phases:
Local Device Assembly Test (LDAT): LDAT testing exercises the functionality and operation of
each standalone device. Devices will be connected to appropriate backhaul for connectivity to the
TCVMS, SCMS, and other cloud services as applicable. An example of CVE LDAT is verifying the
ability of an RSU to broadcast WSAs, be managed through the TCVMS, and request and receive
certificates from the SCMS.
Subsystem Test (SST): SST testing exercises the functionality and operation of each subsystem.
Devices will be connected to form representative subsystems and be connected to appropriate
backhaul for connectivity to the TCVMS, SCMS, and other cloud services as applicable. An example
of CVE SST is verifying the ability of an RSU to broadcast SPaT, MAP, RSMs, SSM, and WSAs, be
managed through the TCVMS, and request and receive certificates from the SCMS.
Final System Test (FST): FST is the final phase of functional and operational testing and serves as
the basis for system acceptance.
Chapter 4. Approach
18 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
Burn-In: Burn-In consists of CVE infrastructure equipment running in “normal” operation mode for a
predetermined amount of time. For the CVE, a 30-day Burn-In period will begin after the FST. Burn-In
demonstrates equipment/system reliability and enables refining of the operation and maintenance
processes prior to installing the full complement of roadside equipment.
Operational Period: The Operational Period begins after system acceptance and continues until the
end of the project. During the Operational Period, the CVE is under operations and maintenance, with
vendors on-call to support as needed and CV data is collected, archived and processes as needed.
4.2.1. First Article Testing
To ensure device functionality and identify configuration parameters before installation, devices are tested in
a controlled environment, which will include a Bench Testing aspect and a Demonstration Area aspect.
These tests will be conducted on “First Article” devices, which are the first devices supplied by vendors that
meet the Smart Columbus CVE requirements.
Bench Testing consists of setting up relevant devices in an indoor controlled lab-type environment in
which devices can be configured and tested to aid in the development of configuration and installation
documentation before installation at the roadside or in vehicles. During Bench Testing, RSUs will be
connected to signal controllers, the SCMS, and the TCVMS to ensure all configuration parameters
are set appropriately to enable operation. OBUs will also be configured to broadcast BSMs and test
connectivity to the SCMS through an RSU. The Traffic Management Center (TMC) will serve as the
location for bench testing. An initial round of LDAT and SST will be completed during Bench Testing.
After the SST is completed, an initial 10-day Burn-In period will begin.
Demonstration Area Testing consists of setting up relevant devices in an outdoor controlled
environment in which vehicles can be driven at appropriate speeds and perform appropriate
maneuvers to test applications. The TMC parking lot will serve as the location for Demonstration Area
Testing. Another round of SST will be completed during Demonstration Area Testing, as this will be
closer to a field deployment setup and configuration than Bench Testing.
Field Testing: After Demonstration Area Testing is completed, First Article devices will be installed at
up to 4 locations along Morse Road and in up to four test vehicles for Field Testing. SST will be
completed after First Article device installation. Once SST is completed, FST will begin on the First
Article installations.
Burn-In: After the FST, the First Article devices will be subjected to a 30-day Burn-In period to
ensure the devices can operate for an extended period.
4.2.2. Failure Reporting
Major and Minor failures that occur during the Burn-In period will be recorded in maintenance logs with the
date, time, and location of the failure, along with the corrective action(s) taken. These logs will be available
for inspection by Smart Columbus during the Burn-In period and provided to the CVE System Owner and
Test Director, hereafter referred to as the Management Team, at the conclusion of the Burn-In period.
A minor failure is defined as a failure that affects the operation of a device but not the operation of the
overall system. During the resolution of a minor failure, the 30-day Burn-In will be suspended. After the
failure is resolved, the Burn-In period will resume. Frequent occurrences of Minor Failures, however, may
indicate a system flaw. If a system flaw is identified, the 30-day Burn-In will reset, after the flaw has been
corrected.
A Major Failure is defined as a failure that affects the entire system, requires more than 48 hours to resolve,
or when less than 95% of CVE equipment under test are operational at any given time. When a Major
Failure occurs, the Management Team will be notified within 48 hours of detection/identification, along with a
Chapter 4. Approach
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 19
Corrective Action Plan. After the Major Failure has been resolved, a Failure Report containing the nature of
the failure, the features/functionality affected by the failure, and the corrective actions taken will be provided
to the Management Team. The 30-day Burn-In period will restart after the resolution of a Major Failure. First
Article testing may be completed in one or more phases, depending on the availability of OBU applications.
4.2.3. Post-Installation Testing
After the 30-day Burn-In period, installation of the remaining infrastructure locations and vehicles will begin
en masse.
Post-Installation Testing consists of verifying the functionality and operation of all devices, immediately
following installation in the field; at infrastructure locations and in vehicles.
For infrastructure locations, the RSU, signal controller, message handler, etc. are tested to ensure proper
integration, operation, and connectivity to the local (intersection) network as well as connectivity to the
TCVMS and SCMS.
For vehicle installations, the OBU is tested to ensure the device is broadcasting BSMs at the appropriate
rate (e.g. 10 times per second), can receive messages from an RSU, and connect to the SCMS through an
RSU.
SST is completed during Post-Installation Testing.
4.2.4. Final System Testing
FST will take place for each corridor in the deployment area. The intent of corridor FST is to ensure
applications are operational throughout the entire length of the corridor; vehicles driving the corridor receive
the appropriate notification/warning. Corridor FST begins after Post-Installation SST is completed for each
infrastructure location along the corridor.
4.2.5. Operational Period
Once Final Testing is completed and the system has been accepted, the CVE Operational Period will begin.
During the Operational Period, RSUs will be monitored for anomalies and repaired as needed and CV data
will be collected and archived as specified.
4.3. TESTING SCHEDULE
Table 3 contains the expected times frame during which the various phases of testing will occur.
Table 3: Test Phase and Timeframes
Test Phase Time Period
First Article Testing Dec. 1-Jan. 30
Bench Testing (LDAT and SST)
Demonstration Area Testing (SST)
Field Testing (SST and FST)
Burn-In
Chapter 4. Approach
20 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
Test Phase Time Period
Remaining Deployment Test Feb. 1-April 21
RSU (SST)
OBU (SST)
Post-Installation Testing April 22-May 19
RSU (SST)
OBU (SST)
Final System Testing (FST) May 20-June 23
Regression June 24 to July 7
Go Live July 7, 2020
Source: City of Columbus
4.4. ROLES
Table 4 contains the anticipated test roles and their responsibility for testing CVE devices before and after
installation.
Table 4: Test Roles and Responsibilities
Title Description Responsibility
CVE Project Manager City of Columbus representative overseeing the completion of the CVE project
Ryan Bollo (Smart City Program Office)
Test Facility Owner Test Facility owners represent the physical location where amenities will be deployed to verify device compliance prior to installation
TMC
OBU Installation Facility Owner
The OBU Installation Facility owner represent the physical location where OBUs are installed in vehicles
Columbus Police (1,3,4,5,6,7,17,18)
Columbus Fire (7,13,16,18,19,24)
Central Ohio Transit Authority (COTA) (McKinley, Fields)
DPS (25th Ave, Morse, Groves)
FCEO (East Maintenance)
Clinton Twp. Fire (61)
Mifflin Twp. Fire (132)
LDV OBU Installers (TBD)
RSU Installation Location Owner
The RSU Installation location owner represents the roadside locations in which RSU and supporting equipment are installed
City of Columbus
ODOT
Franklin County
City of Obetz
Chapter 4. Approach
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 21
Title Description Responsibility
Test Manager Test Manager supervises and controls all tests, reviews and approves test procedures, has the authority to direct all test activities, and is responsible for communicating test status to all stakeholders
Test Manager will notify the City and other key stakeholders of the test schedule at least one week in advance of the scheduled start
Test Manager may not necessarily be present during all test activities but will communicate regularly with the test conductor for status updates
Test Manager signs off on all completed tests based on functionality demonstration
WSP
Test Conductor Test Conductor is responsible for running the daily test activities and demonstrating test results to Test Manager
Kapsch
Danlaw
Siemens
Traffic Data Consumer An authorized individual (or service) that consumes CV data from OBUs and RSUs
TCVMS
Transit Data Consumer Authorized individual (or service) that consumes CV data from transit vehicles
TrCVMS
Data Producer A system or systems that produce data to be consumed
Kapsch
Danlaw
Siemens
Traffic Management Staff
The Traffic Management Staff utilize the TrCVMS to collect data from the CVE
City of Columbus
Transit Management Staff
The Transit Management Staff utilize the TCVMS to collect data from Transit OBUs
COTA
Stakeholders General testing role with domain knowledge in the area of transportation
Safety Manager
Data Recorder Data Recorder is responsible for recording the outputs and overall results of each test
Data Recorder will provide all observations to Test Manager at the end of the test day.
WSP
Chapter 4. Approach
22 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
Title Description Responsibility
Vehicle Drivers Vehicle Drivers will operate vehicles during each applicable test/demonstration. They will follow the explicit instructions in the test script and will be in constant communication with the test conductor to receive further instructions as well as to inform the test conductor of any incidents or exceptions they experience during a test run. Drivers are expected to have a skill level appropriate for testing/demonstrating V2V applications
Siemens
Test Observers Test observers witness test runs at the City's discretion
TBD
Source: City of Columbus
Some roles can be combined such that a single person can assume up to two roles (i.e. the Test
Manager can also be the Data Recorder).
4.5. TEST TOOLS
A key aspect of system testing is the use of both hardware and software tools that enable development,
tracking, and traceability through the process. Operational scenarios from the system ConOps as well as
requirements and design elements specified in the System Design Document form the basis of Chapter 7
Test Scenarios and make up the acceptance criteria for the operational readiness of each RSU location
and OBU equipped vehicle.
This section describes the tools utilized to assess individual components, subsystems and the overall CVE
system.
4.5.1. Connected Vehicle Test Tool
The CV Test Tool (CVTT) is utilized to visualize SPaT, MAP, and RSM messages, verify WSA and BSM,
SRM, SSM, and RTCM message content and capture CV messages for analysis to verify proper operation
of the CVE RSUs and OBUs.
4.5.2. Onboard Unit
The OBU is utilized to demonstrate vehicle-based applications to verify proper operation of the applications
as well as capture CV messages for troubleshooting.
4.5.3. Test Result Log
The CVE project team will rely on tools based on Microsoft Excel to manage the test cases, scenarios, and
test results. The Test Result Log (TRL) is utilized to document the Test Case ID, Test Objective, Number of
Test Runs, and the Test Results for each test. The Test Manager is required to enter the run dates, their
name, result, and comment where applicable. Appendix A.1 includes a sample TRL.
Chapter 4. Approach
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 23
4.5.4. Defect Tracker
A defect tracker is provided for the Test Conductor, which will be used to record anomalies detected during
the execution of a test case or scenario. An observation is considered a defect when the result of an activity
does not match the expected outcomes outlined in the test procedure.
Test Conductors are encouraged to include as much information as possible when recording defects, so
vendors and integrators can use this information (inputs, conditions, step at which failure occurred, etc.) to
try to repeat, identify root case, and resolve the issue. This includes referencing the appropriate test
identifier(s), expected results, actual results, defect frequency (every time, intermittent, etc.). An assessment
of the severity of the defect needs to be performed and assigned as critical, high, medium, low – where
critical is the most serious classification with the feature or product being unusable. Defects of this severity
should be brought to the immediate attention of the Test Manager for further inspection, coordination, and
decision-making. A defect with a low severity indicates the observation is cosmetic in nature.
The Test Manager will monitor the defect log for corrective action. The Test Conductor is responsible for
understanding and reproducing (where possible) the defect, summarizing a response and the activities
taken to resolve the issue, and capturing metadata associated with the resolution (e.g., assigned name,
date, status, description, etc.). If a conflict arises between a design element that ties to a requirement and
the deployed amenity, the Test Manager will coordinate with the appropriate vendor and the system owner
to determine if a change to the design and/or requirement is appropriate. The City of Columbus Project
Manager (who is also the System Owner) will be responsible for reviewing and approving all requests to
make changes that impact the system design and requirements. See Section 4.5.5 for more information
about the Change Request Log.
The defect tracker will also be leveraged to measure the feasibility and readiness of the subcomponents to
be promoted to production. See Appendix A.2 for a sample Defect Tracker.
4.5.5. Change Request Log
The ability to track system design changes or changes to requirements associated with a feature is a
fundamental strategy for configuration management and an important aspect of managing projects and
maintaining traceability across the Smart Columbus program. The Change Request Log provides the Test
Conductor a mechanism to capture and justify requests for change, which often derive from a defect or an
enhancement request. The City of Columbus Project Manager is responsible for assessing the impact of the
change as it relates to the project objectives, schedule, cost, etc., and providing final approval of all change
requests. The Change Request Log in Appendix A.3 provides a record of all the change requests logged
throughout the testing process along with justifications and authorization status.
4.6. ENVIRONMENTAL NEEDS
This section describes the general test environments required to conduct all First Article tests. During
testing, all items within the Test Environment including hardware, software, and configuration elements
under test will be managed by the test team.
4.6.1. Bench Facility
Bench tests involve conducting tests in a controlled environment to assess the initial functionality of the
Smart Columbus CVE OBUs, RSU and supporting equipment. The list below contains general facility and
equipment.
Chapter 4. Approach
24 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
4.6.1.1. GENERAL FACILITIES REQUIREMENTS
The facilities required to conduct the bench tests include:
Workbench space to support four (4) independent test teams
Surge Protection power strips
A Conference Room, with whiteboard and\or flipcharts, etc. large enough to accommodate a
minimum of twelve (12) people
Internet access for up to twelve (12) people
Access to the workbench and conference room space: 8:00 am to 4:30 pm, Monday through Friday
4.6.1.2. GENERAL EQUIPMENT REQUIREMENTS
The equipment required to conduct the bench tests include:
Up to eight (8) RSUs
Up to four OBUs
Four laptops (one per device under test – provided by suppliers)
Hand tools (pliers, screw drivers, socket set, etc.)
Power\battery tools (drill, screwdriver, flashlight, etc.)
110 VAC-to-12V DC power supplies
Assortment of CAT5 Ethernet patch cables (for connecting laptops to devices being tested)
Spectrum Analyzer capable of supporting frequencies up to and including 6 GHz
DSRC Protocol Analyzer capable of decoding and logging WSA, BSM, SPaT, MAP, RTCM, SRM,
SSM, and RSM
Other relevant software tools used during device certification testing
4.6.2. Demonstration Area Facility
The Demonstration Area will be utilized to conduct additional tests not able to be conducted on the bench,
such as the V2V applications. OBUs will be installed in vehicles and Roadside Equipment will be installed at
locations along the drive route.
The list below contains general facility and equipment requirements.
4.6.2.1. GENERAL FACILITIES REQUIREMENTS
The Demonstration Area should include:
Access to power for up to four RSUs, four Signal Controllers, and four other devices etc.
Four vehicles (provided by OBU suppliers)
Space to set up four independent RSU Test Stations
Space to drive vehicles to test V2V applications
Parking lot space accommodating up to eight LDVs simultaneously
Chapter 4. Approach
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 25
A Conference Room large enough to accommodate a minimum of 12 people
Internet access for up to 12 people
Access to the facility: 8 a.m. to 4:30 p.m., Monday through Friday
4.6.2.2. GENERAL EQUIPMENT REQUIREMENTS
The equipment required to conduct the field tests include:
Up to eight RSUs
Up to four OBUs
Four laptops (one per device under test – provided by the suppliers)
Assortment of CAT5 Ethernet patch cables (for connecting laptops to devices being tested)
Spectrum Analyzer capable of supporting frequencies up to and including 6 GHz
DSRC Protocol Analyzer capable of decoding and logging WSA, BSM, SPaT, MAP, RTCM, SRM,
SSM, and RSM
Other relevant tools used during device certification testing
4.6.2.3. VEHICLES
The OBU vendor(s) will provide vehicles and Vehicle Drivers to conduct V2V application testing and
demonstration. At a minimum, the following are required to demonstrate V2V applications.
Two LDVs
One OBU installed in each vehicle
One driver for each vehicle
It is assumed the Smart Columbus Program Management Office will provide the facilities required to
conduct both the First Article and Demonstration Area tests. The test team will provide the core test team
members, coordinate with various stakeholder personnel to attend relevant tests, conduct all tests, and
provide test results to Smart Columbus for review.
4.6.3. Test Preparation
Prior to conducting tests, the test environment will be inspected to ensure all hardware and software
required to execute each test are readily available.
For First Article testing, a site survey will be conducted of the applicable test facility(ies) to verify tools and
space are available as well as connectivity to the SCMS, the Traffic Connected Vehicle Management
System (TCVMS), and the RTCM source prior to setting up devices. A spectrum analysis may also be
conducted to verify the test facility is free of interfering RF transmissions in the 5.9 GHz, and neighboring
spectrum, as interference can adversely affect the operations of DSRC devices.
For Demonstration Area testing, a site survey will be conducted to verify space is available as well as
connectivity to the SCMS and the TCVMS prior to setting up the RSUs and Signal Controllers. A spectrum
analysis may be conducted to verify the Demonstration Area is free of interfering RF transmissions in the 5.9
GHz, and neighboring spectrum, as interference can adversely affect the operations of the DSRC devices.
Chapter 4. Approach
26 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
4.7. MEASURES AND METRICS
The City of Columbus will utilize Microsoft Excel to capture anomalies, incongruences, errors, or any other
output inconsistent with the expected test case result. These tools will capture the following testing metrics:
Total number of test cases
Number of test runs per case
Number and percentage of test cases passed
Number and percentage of test cases failed
Number and percentage of test cases deferred
Number and percentage of defects found (relative to total cases)
Number and percentage of high severity defects
Number and percentage of defects accepted
Number and percentage of defects rejected
Number and percentage of defects deferred
Total number of testers
A high-level Test Summary Report will be developed to track the overall number of Test Cases executed as
well as the number that have passed and failed. A high-level Defect Matrix will be developed to track the
number of defects opened, closed, canceled, resoled, and deferred.
The City of Columbus will leverage these data points to determine the feasibility and operational readiness
of the CVE to receive final acceptance test approval. See Appendix A.4 for a sample Test Summary Report
and Defect Matrix.
4.8. TEST CRITERIA
4.8.1. Item Pass/Fail
Each test case consists of several unique properties which should be considered holistically during the
testing evaluation process. Properties include but are not limited to: test identifier (ID), test objective,
expected outcome, number of test runs that must be completed, and status.
Planned: The test case has been defined, role identified, testers assigned, and is ready for testing.
In Progress: The test case is underway but has not been completed.
Pass: A pass value indicates tests have completed the defined number of runs by various testers
without error and the expected result has been achieved. It is expected that each time this test is
performed, independent of who is testing, the same successful results will be achieved. There may
be instances when a tester identifies a defect during the procedure, yet the test case still achieves the
stated outcome. The case can still pass, but the testers must log the defect and bring it to the
attention of the test manager. This can happen when there are minor bugs, not critical to essential
functionality of the feature being tested, such as an image being out of alignment or a misspelling.
Fail: A test case is marked as failed when part or all the expected outcome is not met. When a defect
is detected/observed it will be assigned a Defect ID, for traceability, and a description of the results
will be logged for comparison after treatment. All failed test cases must include one or more defects
to capture the details surrounding the failure and to track its status.
Chapter 4. Approach
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 27
Deferred: A test case is marked as deferred when the case is unable to be performed at the current
time of testing or when there is a change in requirements. Most often this will occur when a software
product is being released in increments and the functionality is not ready when it’s time to test the
current release. This also applies to any features the system may include where testing will be
performed outside of the scope of this Master Test Plan (MTP). If a test is deferred, the Test Manager
should provide a brief reason for the delay. The Test Manager is responsible for tracking deferred
cases and evaluating the most appropriate time and/or response for addressing the case.
See for a sample Appendix A.5 Test Case Status form.
4.8.2. Testing Suspension and Resumption
There are cases when a critical, severe defect is detected that is significant enough that – if not addressed –
would require one or more iterations of the same tests to be performed again. In these situations, it is better
to suspend testing until the defect is resolved to prevent wasting the project budget and the testers’ time.
The Test Manager should be notified immediately and work with vendors and other appropriate stakeholders
to correct the issue as quickly as possible. Testing will resume once the test manager has successfully
confirmed the issue has been resolved.
The following situations would cause testing to be suspended:
1. One or more defects found associated with the RSU’s or OBU’s ability to broadcast messages.
2. One or more defects found associated with the OBU’s ability to provide appropriate notification/warning
to the driver.
Communication network failure rendering RSUs isolated from the TCVMS, SCMS or other cloud services
required for normal operation.
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 29
Chapter 5. Test Deliverables
During testing there will be several artifacts utilized to support and enhance the testing process. The
following artifacts make up part of this testing plan:
Test cases
Test scenarios
Testing matrix
Defects matrix with corrective actions
Change request log
Error logs, bug reports, and/or screen captures (where feasible)
Acceptance (see Appendix A)
CVE Test Report webinar
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 31
Chapter 6. Test Cases
The test cases designed for this MTP will focus on testing the system requirements, interfaces, data, and system design for the CVE system. The scenarios will expand on these essential functions to
test the system holistically. The number of testers shown in the tables below were derived by the type of testers that would be knowledgeable in the test objective and how important the objective is to
the overall CVE Project. The following test cases are planned as outlined.
6.1. ONBOARD UNIT
Table 5: Onboard Unit (Test Cases)
Test Case ID Test Type Function Verification Method Test Objective
CVE Test Locations Tester Role Metric Pass Criteria
CVE-OBU001-v01
First Article
Post-Installation
BSM Broadcast Demonstration Verify OBUs broadcast valid BSMs 10 times per second.
Bench/Vehicle installation site
Test Conductor
Data Producer
Data Recorder
OBUs broadcast BSMs at a rate of 10/sec
Data Recorder observes OBUs broadcasting 10 BSMs per second
CVE-OBU002-v01
First Article Transit Vehicle OBU –FCW Event Capture Trigger
Demonstration Verify a host Transit Vehicle OBU logs an FCW-based transit vehicle interaction event only when an FCW threat exists
Demonstration Area Test Conductor
Data Producer
Data Recorder
Transit Vehicle OBU logs an FCW event, under appropriate vehicle maneuver
Data Recorder observes FCW event in log file, after appropriate vehicle maneuver
CVE-OBU003-v01
First Article Transit Vehicle OBU –EEBL Event Capture Trigger
Demonstration Verify a host Transit Vehicle OBU logs an EEBL-based interaction event only when EEBL threat exists
Demonstration Area Test Conductor
Data Producer
Data Recorder
Transit Vehicle OBU logs an EEBL event, under appropriate vehicle maneuver
Data Recorder observes EEBL event in log file, after appropriate vehicle maneuver
Chapter 6. Test Cases
32 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
Test Case ID Test Type Function Verification Method Test Objective
CVE Test Locations Tester Role Metric Pass Criteria
CVE-OBU004-v01
First Article Transit Vehicle OBU –IMA Event Capture Trigger
Demonstration Verify a host Transit Vehicle OBU logs an IMA-based transit vehicle interaction event only when an IMA threat exists
Demonstration Area Test Conductor
Data Producer
Data Recorder
Transit Vehicle OBU logs an IMA event, under appropriate vehicle maneuver
Data Recorder observes IMA event in log file, after appropriate vehicle maneuver
CVE-OBU005-v01
First Article Transit Vehicle OBU – LCW Event Capture Trigger
Demonstration Verify a host Transit Vehicle OBU logs an LCW-based transit vehicle interaction event only when an LCW threat exists
Demonstration Area Test Conductor
Data Producer
Data Recorder
Transit Vehicle OBU logs an LCW event, under appropriate vehicle maneuver
Data Recorder observes LCW event in log file, after appropriate vehicle maneuver
CVE-OBU006-v01
First Article Transit Vehicle OBU – RLVW Event Capture Trigger
Demonstration Verify a host Transit Vehicle OBU logs an RLVW-based transit vehicle interaction event only when an RLVW threat exists
Demonstration Area Test Conductor
Data Producer
Data Recorder
Transit Vehicle OBU logs an RLVW event, if the vehicle is approaching the intersection in such a way that the vehicle will run a Red Light
Data Recorder observes RLVW event in log file, after appropriate vehicle maneuver
Chapter 6. Test Cases
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 33
Test Case ID Test Type Function Verification Method Test Objective
CVE Test Locations Tester Role Metric Pass Criteria
CVE-OBU007-v01
First Article Transit Vehicle OBU – RSSZ Event Capture Trigger
Demonstration Verify a host Transit Vehicle OBU logs an RSSZ-based transit vehicle interaction event only when receiving a RSSZ message from an RSU and the host vehicle is in, or approaching, the School Zone traveling above the School Zone speed limit
School Zone Demonstration Area
Test Conductor
Data Producer
Data Recorder
Transit Vehicle OBU logs an RSSZ event, if the vehicle is traveling above the RSSZ Speed Limit in the RSSZ Zone
Data Recorder observes RSSZ event in log file, after the vehicle travers the RSSZ traveling above the RSSZ Speed Limit
CVE-OBU008-v01
First Article Transit Vehicle OBU – TSP Event Capture Trigger
Demonstration Verify a host Transit Vehicle OBU logs a TSP-based transit vehicle interaction event when it broadcasts an SRM
Demonstration Area Test Conductor
Data Producer
Data Recorder
Transit Vehicle OBU logs an TSP event, when the OBUs requests Signal Priority
Data Recorder observes TSP event in log file, after the OBU requests Signal Priority
CVE-OBU009-v01
First Article Transit Vehicle OBU – Temporary Message Logging
Demonstration Verify a Transit Vehicle OBU logs all messages (BSM, SPaT, MAP, SRM, SSM, RTCM, RSM) that it receives for a pre-determined period
Demonstration Area Test Conductor
Data Producer
Data Recorder
Transit Vehicle OBU log files only contain received messages for a given period (e.g. 5 seconds)
Data Recorder observes OBU log file only contains messages for a predetermined amount of time (e.g. the last 5 minutes of testing)
Chapter 6. Test Cases
34 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
Test Case ID Test Type Function Verification Method Test Objective
CVE Test Locations Tester Role Metric Pass Criteria
CVE-OBU010-v01
First Article Transit Vehicle OBU – Associating Logged Messages with a Transit Vehicle Interaction Event
Demonstration Verify a Transit Vehicle Interaction Event consists of all logged messages for a predetermined period before and after the warning generating the event was issued.
Demonstration Area Test Conductor
Data Producer
Data Recorder
Transit Vehicle OBU logs received message for a predetermined amount of time before and after an "event" (e.g. 3 seconds before and after an event).
Data Recorder observes OBU log file contains messages for a predetermined amount of time before and after an "event" (e.g. 3 seconds before and after an event).
CVE-OBU011-v01
First Article Transit Vehicle OBU – Upload Transit Vehicle Interaction Event Data
Demonstration Verify a Transit Vehicle OBU uploads Transit Vehicle Interaction Event Data to the TrCVMS when arriving at the COTA garage, and deletes local transit vehicle interaction events after uploading
COTA McKinley Ave Garage, COTA Fields Ave Garage
Test Conductor
Data Producer
Data Recorder
Transit Data Consumer
Transit Vehicle OBU uploads OBU log files to TrCVMS and deletes uploaded log files from internal storage. TrCVMS archives OBU log files
Data Recorder observes TrCVMS archive contains OBU log files with current date and time stamp (of test) and OBU does not contain log files.
CVE-OBU012-v01
First Article EV SRM Broadcast Demonstration Verify the EV OBU broadcasts SRMs only when lights and sirens are active, and it is approaching an equipped intersection
Priority/Preemption Demonstration Area
Test Conductor
Data Producer
Data Recorder
EV OBUs broadcasts SRMs only when the vehicles lights and sirens are on
Data Recorder observes SRMs are only broadcast when the EV’s lights and sirens are on
CVE-OBU013-v01
First Article EV SSM Receipt Demonstration Verify the EV OBU receives an SSM
Priority/Preemption Demonstration Area
Test Conductor
Data Producer
Data Recorder
EV OBU logs received SSMs
Data Recorder observes OBU log file contains SSMs after an SRM is issued
Chapter 6. Test Cases
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 35
Test Case ID Test Type Function Verification Method Test Objective
CVE Test Locations Tester Role Metric Pass Criteria
CVE-OBU014-v01
First Article Transit Vehicle SRM Broadcast
Demonstration Verify the Transit Vehicle OBU broadcasts SRMs only when approaching an equipped intersection
Priority/Preemption Demonstration Area
Test Conductor
Data Producer
Data Recorder
Transit Vehicle OBU only broadcasts SRMs when in range of an RSU broadcasting WSAs containing PSID 0x20-40-96
Data Recorder observes OBU log file contains SRMs only when WSA's containing PSDI 0x02-40-96 are received
CVE-OBU015-v01
First Article Transit Vehicle SSM Receipt
Demonstration Verify the Transit Vehicle OBU receives an SSM
Priority/Preemption Demonstration Area
Test Conductor
Data Producer
Data Recorder
Transit Vehicle OBU logs received SSMs
Data Recorder observes OBU log file contains SSMs after an SRM is issued
CVE-OBU016-v01
First Article HDV SRM Broadcast
Demonstration Verify the HDV OBU broadcasts SRMs only when approaching an equipped intersection
Priority/Preemption Demonstration Area
Test Conductor
Data Producer
Data Recorder
Heavy Duty Vehicle OBU only broadcasts SRMs when in range of an RSU broadcasting WSAs containing PSID 0x20-40-96
Data Recorder observes OBU log file contains SRMs only when WSA's containing PSDI 0x02-40-96 are received
CVE-OBU017-v01
First Article HDV SSM Receipt Demonstration Verify the HDV OBU receives an SSM
Priority/Preemption Demonstration Area
Test Conductor
Data Producer
Data Recorder
Heavy Duty Vehicle OBU logs received SSMs
Data Recorder observes OBU log file contains SSMs after an SRM is issued
Source: City of Columbus
Chapter 6. Test Cases
36 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
6.2. ONBOARD UNIT APPLCIATION
Table 6: Onboard Unit Application Test Cases
Test Case ID Test Type Function Verification Method Test Objective
CVE Test Locations Tester Role Metric Pass Criteria
CVE-App001-v01 First Article LDV OBU – Emergency Electronic Brake Light Warning Trigger
Demonstration Verify a host LDV OBU issues an EEBL warning when it receives a BSM from a remote OBU (under conditions in which an EEBL warning should be issued)
Demonstration Area Test Conductor
Data Producer
Vehicle Drivers
Data Recorder
Light Duty Vehicle OBU issues an EEBL warning only under appropriate vehicle maneuvers
Data Recorder observes EEBL warning
CVE-App002-v01 First Article LDV OBU – Forward Collision Warning Trigger
Demonstration Verify a host LDV OBU issues an FCW warning when it receives a BSM from a remote OBU (under conditions in which an FCW warning should be issued)
Demonstration Area Test Conductor
Data Producer
Vehicle Drivers
Data Recorder
Light Duty Vehicle OBU issues an FCW warning only under appropriate vehicle maneuvers
Data Recorder observes FCW warning
CVE-App003-v01 First Article LDV OBU – Lane Change Warning Trigger
Demonstration Verify a host LDV OBU issues an LCW warning when it receives a BSM from a remote OBU (under conditions in which an LCW warning should be issued)
Demonstration Area Test Conductor
Data Producer
Vehicle Drivers
Data Recorder
Light Duty Vehicle OBU issues an LCW warning only under appropriate vehicle maneuvers
Data Recorder observes LCW warning
Chapter 6. Test Cases
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 37
Test Case ID Test Type Function Verification Method Test Objective
CVE Test Locations Tester Role Metric Pass Criteria
CVE-App004-v01 First Article LDV OBU – Intersection Movement Assist Warning Trigger
Demonstration Verify a host LDV OBU issues an IMA warning when it receives a BSM from a remote OBU (under conditions in which an IMA warning should be issued)
Demonstration Area Test Conductor
Data Producer
Vehicle Drivers
Data Recorder
Light Duty Vehicle OBU issues an IMA warning only under appropriate vehicle maneuvers
Data Recorder observes IMA warning
CVE-App005-v01 First Article LDV OBU – Red Light Violation Warning Trigger
Demonstration Verify a host LDV OBU issues an RLVW warning when it receives SPaT and MAP from an RSU (under conditions in which an RLVW warning should be issued)
Demonstration Area Test Conductor
Data Producer
Vehicle Drivers
Data Recorder
Light Duty Vehicle OBU issues an RLVW warning only if the vehicle is approaching the intersection in such a way that the vehicle will run a Red Light
Data Recorder observes RLVW warning
CVE-App006-v01 First Article LDV OBU – Reduced Speed School Zone Warning Trigger
Demonstration Verify a host LDV OBU issues an RSSZ warning when it receives a RSM from a RSU (under conditions in which an RSSZ warning should be issued)
School Zone Demonstration Area
Test Conductor
Data Producer
Vehicle Drivers
Data Recorder
Light Duty Vehicle OBU issues an RSSZ warning only if the vehicle is traveling above the RSSZ Speed Limit in the RSSZ zone
Data Recorder observes vehicles speed and RSSZ warning
CVE-App007-v01 First Article EV Driver Notification
Demonstration Verify the EV OBU provides notification to driver when preemption is granted
Priority/Preemption Demonstration Area
Test Conductor
Data Producer
Vehicle Drivers
Data Recorder
EV OBU issues a notification when a Signal Preemption Request is granted
Data Recorder observes the Preemption notification
Chapter 6. Test Cases
38 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
Test Case ID Test Type Function Verification Method Test Objective
CVE Test Locations Tester Role Metric Pass Criteria
CVE-App008-v01 First Article LDV OBU – Warning Arbitration
Demonstration Verify a high priority warning takes precedence over a low priority warning when both warning types are present.
Demonstration Area Test Conductor
Data Producer
Vehicle Drivers
Data Recorder
Light Duty Vehicle OBU issues the high priority warning when both a high and low priority event exists
Data Recorder observes high priority warning
Source: City of Columbus
6.3. ROADSIDE UNIT
Table 7: Roadside Unit Test Cases
Test Case ID Test Type Function Verification Method Test Objective
CVE Test Locations Tester Role Metric Pass Criteria
CVE-RSU001-v01 First Article
Post-Installation
VDTO – BSM Forwarding
Demonstration Verify RSU forwards BSMs, received from passing vehicles, to TCVMS.
Demonstration Area
All RSU Installation Locations
Test Conductor
Data Producer
Data Recorder
RSU forwards BSMs to TCVMS
Data Recorder observes BSMs from applicable RSU in the TCVMS
CVE-RSU002-v01 First Article
Post-Installation
Signal Priority – SRM forwarding
Demonstration Verify RSU forwards SRMs, received from approaching vehicles, to the message handler
Demonstration Area
All RSU Installation Locations
Test Conductor
Data Producer
Data Recorder
RSU forwards SRMs to the local message handler
Data Recorder observes SRMs being sent to the message handler; either through inspection of the message handler log file (post processing) or through a message handler engineering interface in real-time
Chapter 6. Test Cases
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 39
Test Case ID Test Type Function Verification Method Test Objective
CVE Test Locations Tester Role Metric Pass Criteria
CVE-RSU003-v01 First Article
Post-Installation
SPaT Broadcast Demonstration Verify RSUs broadcast SPaT messages received from the Message Handler over DSRC on Channel 172 at 10Hz
Demonstration Area
All RSU Installation Locations
Test Conductor
Data Producer
Data Recorder
RSU broadcast SPaT messages
Data Recorder observes the RSU broadcasting SPaT messages; either through inspection of the RSU log file (post processing) or through a live over-the-air data capture tool
CVE-RSU004-v01 First Article
Post-Installation
SPaT Broadcast Demonstration Verify SPaT broadcast by RSUs match the Signal Head Status
Demonstration Area
All RSU Installation Locations
Test Conductor
Data Producer
Data Recorder
SPaT status matches Signal Head status
Data Recorder observes the SPaT status presented on the CVTT matches the status of the Signal Head
CVE-RSU005-v01 First Article
Post-Installation
MAP Broadcast Demonstration Verify RSUs store MAP files received from the TCVMS
Demonstration Area
All RSU Installation Locations
Test Conductor
Data Producer
Data Recorder
RSU stores MAP files
Data Recorder observes MAP files stored on the RSU
CVE-RSU006-v01 First Article
Post-Installation
MAP Broadcast Demonstration Verify RSUs broadcast MAP messages over DSRC on Channel 172 at 1Hz
Demonstration Area
All RSU Installation Locations
Test Conductor
Data Producer
Data Recorder
RSU broadcast MAP messages
Data Recorder observes the RSU broadcasting MAP messages; either through inspection of the RSU log file (post processing) or through a live over-the-air data capture tool
Chapter 6. Test Cases
40 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
Test Case ID Test Type Function Verification Method Test Objective
CVE Test Locations Tester Role Metric Pass Criteria
CVE-RSU007-v01 First Article
Post-Installation
MAP Demonstration Verify MAP file matches intersection geometry
Demonstration Area
All RSU Installation Locations
Test Conductor
Data Producer
Data Recorder
Decoded MAP message broadcast by the RSU matches the applicable intersection geometry
Data Recorder observes decoded MAP message overlaid on Google Maps, or similar Map tool
CVE-RSU008-v01 First Article
Post-Installation
RTCM Broadcast Demonstration Verify RSUs broadcast RTCM messages received from the Message Handler over DSRC on the specified Service Channel at the specified rate
Demonstration Area
All RSU Installation Locations
Test Conductor
Data Producer
Data Recorder
RSU broadcast RTCM messages
Data Recorder observes the RSU broadcasting RTCM messages; either through inspection of the RSU log file (post processing) or through a live over-the-air data capture tool
CVE-RSU009-v01 First Article
Post-Installation
Signal Priority – SSM Broadcast
Demonstration Verify RSUs broadcasts SSM received from the Message Handler over DSRC on the specified Service Channel at the specified rate
Demonstration Area
RSUs supporting Priority/Preemption
Test Conductor
Data Producer
Data Recorder
RSU broadcast SSM messages
Data Recorder observes the RSU broadcasting SSM messages; either through inspection of the RSU log file (post processing) or through a live over-the-air data capture tool
Chapter 6. Test Cases
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 41
Test Case ID Test Type Function Verification Method Test Objective
CVE Test Locations Tester Role Metric Pass Criteria
CVE-RSU010-v01 First Article
Post-Installation
School Zone – RSM Broadcast
Demonstration Verify RSUs broadcasts RSMs at 1 Hz when the School Zone signal is flashing.
Demonstration Area
RSUs supporting RSSZ
Test Conductor
Data Producer
Data Recorder
RSU broadcast RSSZ based RSM messages when the local "School Zone signal" is flashing
Data Recorder observes the RSU broadcasting RSMs only when the local "School Zone signal" is flashing; either through inspection of the RSU log file (post processing) or through a live over-the-air data capture tool
CVE-RSU011-v01 First Article
Post-Installation
System Monitoring Demonstration Verify RSUs report Channel Busy Ratio to TCVMS, when above a predetermined threshold
Demonstration Area
All RSU Installation Locations
Test Conductor
Data Producer
Data Recorder
RSU sends "channel busy ratio" (CBR) parameters to the TCVMS only when the CBR is above a specified threshold (e.g. 80%)
Data Recorder observes that only CBR values above a given threshold (e.g. 80%) are logged in the TCVMS
CVE-RSU012-v01 First Article
Post-Installation
WSA Broadcast Demonstration Verify RSU broadcast WSA with correct parameters (including WRA)
Demonstration Area
All RSU Installation Locations
Test Conductor
Data Producer
Data Recorder
WSAs broadcast by an RSU contains the correct parameters
Data Recorder observes WSAs broadcast by the RSU contain the correct parameters; either through inspection of the RSU log file (post processing) or through a live over-the-air data capture tool
Chapter 6. Test Cases
42 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
Test Case ID Test Type Function Verification Method Test Objective
CVE Test Locations Tester Role Metric Pass Criteria
CVE-RSU013-v01 First Article
Post-Installation
IPv6 Services Demonstration Verify IPv6 connectivity to SCMS
Demonstration Area
All RSU Installation Locations
Test Conductor
Data Producer
Data Recorder
OBUs are able to connect to the SCMS through the RSU
Data Recorder observes OBUs are able to connect to the SCMS through the RSU; either through inspection of the OBU log file (post processing) or through a live over-the-air data capture tool
Source: City of Columbus
6.4. MESSAGE HANDLER
Table 8: Message Handler Test Cases
Test Case ID Test Type Function Verification Method Test Objective
CVE Test Locations Tester Role Metric Pass Criteria
CVE-MH001-v01 First Article
Post-Installation
SPaT message generation
Test Verify the Message Handler generates and forwards SAE J2735-2016 SPaT messages, based on SPaT data received from the local signal controller, to the local RSU
Demonstration Area
All RSU Installation Locations
Test Conductor
Data Producer
Data Recorder
Message Handler generates J2735 SPaT Messages
Data Recorder observes Message Handler sending J2735 SPaT messages to the local RSU; either through inspection of the Message Handler log file (post processing) or through a Message Handler engineering interface in real-time
Chapter 6. Test Cases
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 43
Test Case ID Test Type Function Verification Method Test Objective
CVE Test Locations Tester Role Metric Pass Criteria
CVE-MH002-v01 First Article
Post-Installation
Signal Priority – Authorization
Test Verify the Message Handler properly determines if OBU requesting priority/preemption is authorized to receive priority/preemption
Demonstration Area
Locations supporting Priority/Preemption
Test Conductor
Data Producer
Data Recorder
Message Handler verifies (via 1609.2 certificates) Signal Priority/Preemption requests from RSU
Data Recorder observes the Message Handler verifying incoming SRMs; either through inspection of the Message Handler log file (post processing) or through a Message Handler engineering interface in real-time
CVE-MH003-v01 First Article
Post-Installation
Signal Priority – Arbitration
Test Verify the Message Handler properly arbitrates priority/preemption requests when receiving multiple valid requests.
Demonstration Area
Locations supporting Priority/Preemption
Test Conductor
Data Producer
Data Recorder
Message Handler only places a Priority/Preemption call for the highest priority request; A Preemption request is higher priority than a Priority request
Data Recorder observes the Message Handler sending the highest priority request to the Signal controller; either through inspection of the Message Handler log file (post processing) or through a Message Handler engineering interface in real-time
Chapter 6. Test Cases
44 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
Test Case ID Test Type Function Verification Method Test Objective
CVE Test Locations Tester Role Metric Pass Criteria
CVE-MH004-v01 First Article
Post-Installation
Signal Priority – Place call to signal controller
Test Verify the Message Handler places a priority call to the local signal controller based on an authorized SRM.
Demonstration Area
Locations supporting Priority/Preemption
Test Conductor
Data Producer
Data Recorder
Message Handler places a Priority Call based on received SRM
Data Recorder observes the result of the Message Handler sending a Priority Call to the Signal Controller; either through inspection of the Message Handler log file (post processing) or through a Message Handler engineering interface in real-time
CVE-MH005-v01 First Article
Post-Installation
Signal Preemption – Place call to signal controller
Test Verify the Message Handler places a preemption call to the local signal controller based on an authorized SRM.
Demonstration Area
Locations supporting Priority/Preemption
Test Conductor
Data Producer
Data Recorder
Message Handler places a Preemption Call based on received SRM
Data Recorder observes the Message Handler sending a Preemption Call to the Signal Controller; either through inspection of the Message Handler log file (post processing) or through a Message Handler engineering interface in real-time
Chapter 6. Test Cases
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 45
Test Case ID Test Type Function Verification Method Test Objective
CVE Test Locations Tester Role Metric Pass Criteria
CVE-MH006-v01 First Article
Post-Installation
Signal Priority – SSM generation
Test Verify the Message Handler generates and forwards SAE J2735-2016 SSMs, based on status data received from the local Signal Controller, to the local RSU
Demonstration Area
Locations supporting Priority/Preemption
Test Conductor
Data Producer
Data Recorder
Message Handler sends SSM to RSU
Data Recorder observes the Message Handler sending an SSM to the RSU; either through inspection of the Message Handler log file (post processing) or through a Message Handler engineering interface in real-time
CVE-MH008-v01 First Article
Post-Installation
RTCM message generation
Test Verify the Message Handler generates and forwards SAE J2735-2016 RTCM messages, based on data received from the Position Correction System
Demonstration Area
All RSU Installation Locations
Test Conductor
Data Producer
Data Recorder
Message Handler sends RTCM to RSU
Data Recorder observes the Message Handler sending a RTCMs to the RSU; either through inspection of the Message Handler log file (post processing) or through a Message Handler engineering interface in real-time
Source: City of Columbus
Chapter 6. Test Cases
46 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
6.5. TRAFFIC SIGNAL CONTROLLER
Table 9: Traffic Signal Controller Test Cases
Test Case ID Test Type Function Verification Method Test Objective
CVE Test Locations Tester Role Metric Pass Criteria
CVE-TSC001-v01
First Article
Post-Installation
SPaT data Test Verify Traffic Signal Controller sends SPaT data to the Message Handler
Demonstration Area
All RSU Installation Locations
Test Conductor
Data Producer
Data Recorder
Signal Controller sends SPaT data to Message Handler
Data Recorder observes the Signal Controller sending data to the Message Handler; either through inspection of the Message Handler log file (post processing) or through a Message Handler engineering interface in real-time
CVE-TSC002-v01
First Article
Post-Installation
Signal Priority call Test Verify Traffic Signal Controller processes a Signal Priority Call from the Message Handler
Demonstration Area
All RSU Installation Locations
Test Conductor
Data Producer
Data Recorder
Signal Controller modifies Signal Timing based on a Priority Call from the Message Handler
Data Recorder observes the Signal Controller changing the signal timing through the Signal Controller User Interface
CVE-TSC003-v01
First Article
Post-Installation
Signal Preemption call
Test Verify Traffic Signal Controller processes a Signal Preemption Call from the Message Handler
Demonstration Area
All RSU Installation Locations
Test Conductor
Data Producer
Data Recorder
Signal Controller modifies Signal Timing based on a Preemption Call from the Message Handler
Data Recorder observes the Signal Controller changing the signal timing through the Signal Controller User Interface
Chapter 6. Test Cases
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 47
Test Case ID Test Type Function Verification Method Test Objective
CVE Test Locations Tester Role Metric Pass Criteria
CVE-TSC004-v01
First Article
Post-Installation
Signal Priority status Test Verify Traffic Signal Controller sends Priority status to the Message Handler
Demonstration Area
All RSU Installation Locations
Test Conductor
Data Producer
Data Recorder
Signal Controller sends Signal Priority status data to Message Handler
Data Recorder observes the Signal Controller sending Signal Priority Status data to the Message Handler; either through inspection of the Message Handler log file (post processing) or through a Message Handler engineering interface in real-time
CVE-TSC005-v01
First Article
Post-Installation
Signal Preemption status
Test Verify Traffic Signal Controller sends Preemption status to the Message Handler
Demonstration Area
All RSU Installation Locations
Test Conductor
Data Producer
Data Recorder
Signal Controller sends Signal Preemption status data to Message Handler
Data Recorder observes the Signal Controller sending Signal Preemption Status data to the Message Handler; either through inspection of the Message Handler log file (post processing) or through a Message Handler engineering interface in real-time
Source: City of Columbus
Chapter 6. Test Cases
48 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
6.6. TRAFFIC CV MANAGEMENT SYSTEM
Table 10: Traffic CV Management System Test Cases
Test Case ID Test Type Function Verification Method Test Objective
CVE Test Locations Tester Role Metric Pass Criteria
CVE-TCVMS001-v01
First Article MAP Input Demonstration Verify Traffic Management Staff can input a MAP message payload to the TCVMS and specify which RSU it should be broadcast from
Traffic Management Center
Test Conductor
Data Producer
Traffic Data Consumer
Data Recorder
Traffic Management Staff sends a MAP file to an RSU through the TCVMS User Interface
Data Recorder observes Traffic Management Staff uploading a MAP file to an RSU through the TCVMS User Interface and inspecting the RSU MAP File storage directory
CVE-TCVMS002-v01
First Article RSM Input Demonstration Verify Traffic Management Staff can input an RSM payload to the TCVMS and specify which RSU it should be broadcast from
Traffic Management Center
Test Conductor
Data Producer
Traffic Data Consumer
Data Recorder
Traffic Management Staff sends an RSM file to an RSU through the TCVMS User Interface
Data Recorder observes Traffic Management Staff uploading an RSM file to an RSU through the TCVMS User Interface and inspecting the RSU RSM File storage directory
CVE-TCVMS003-v01
First Article Receive/Archive MAP
Demonstration Verify the TCVMS archives the MAP message locally and forwards it to the Operating System
Traffic Management Center
Test Conductor
Data Producer
Traffic Data Consumer
Data Recorder
TCVMS archives MAP file
Data Recorder observes Archived MAP files through inspection of the TCVMS Archive Data storage folder
Chapter 6. Test Cases
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 49
Test Case ID Test Type Function Verification Method Test Objective
CVE Test Locations Tester Role Metric Pass Criteria
CVE-TCVMS004-v01
First Article Receive/Archive RSM
Demonstration Verify the TCVMS archives the RSM locally and forwards it to the Operating System
Traffic Management Center
Test Conductor
Data Producer
Traffic Data Consumer
Data Recorder
TCVMS archives RSM file
Data Recorder observes Archived RSM files through inspection of the TCVMS Archive Data storage folder
CVE-TCVMS005-v01
First Article Receive/Archive BSM
Demonstration Verify the TCVMS archives the BSM locally and forwards it to the Operating System
Traffic Management Center
Test Conductor
Data Producer
Traffic Data Consumer
Data Recorder
TCVMS archives BSMs and send BSMs to the Operating System
Data Recorder observes Archived BSMs through inspection of the TCVMS Archive Data storage folder and either through inspection of the TCVMS log file (post processing) or through a TCMS engineering interface in real-time
CVE-TCVMS006-v01
First Article Receive/Archive SPaT
Demonstration Verify the TCVMS archives the SPaT Message locally and forwards it to the Operating System
Traffic Management Center
Test Conductor
Data Producer
Traffic Data Consumer
Data Recorder
TCVMS archives SPaT messages and sends SPaT message to the Operating System
Data Recorder observes Archived SPaT messages through inspection of the TCVMS Archive Data storage folder and either through inspection of the TCVMS log file (post processing) or through a TCVMS engineering interface in real-time
Chapter 6. Test Cases
50 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
Test Case ID Test Type Function Verification Method Test Objective
CVE Test Locations Tester Role Metric Pass Criteria
CVE-TCVMS007-v01
First Article Receive/Archive SRM
Demonstration Verify the TCVMS archives the SRM locally and forwards it to the Operating System
Traffic Management Center
Test Conductor
Data Producer
Traffic Data Consumer
Data Recorder
TCVMS archives SRM messages and sends SRM message to the Operating System
Data Recorder observes Archived SRM messages through inspection of the TCVMS Archive Data storage folder and either through inspection of the TCVMS log file (post processing) or through a TCVMS engineering interface in real-time
CVE-TCVMS008-v01
First Article Receive/Archive SSM
Demonstration Verify the TCVMS archives the SSM locally and forwards it to the Operating System
Traffic Management Center
Test Conductor
Data Producer
Traffic Data Consumer
Data Recorder
TCVMS archives SSM messages and sends SSM message to the Operating System
Data Recorder observes Archived SSM messages through inspection of the TCVMS Archive Data storage folder and either through inspection of the TCVMS log file (post processing) or through a TCVMS engineering interface in real-time
Chapter 6. Test Cases
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 51
Test Case ID Test Type Function Verification Method Test Objective
CVE Test Locations Tester Role Metric Pass Criteria
CVE-TCVMS009-v01
First Article Receive/Archive RTCM
Demonstration Verify the TCVMS archives the RTCM locally and forwards it to the Operating System
Traffic Management Center
Test Conductor
Data Producer
Traffic Data Consumer
Data Recorder
TCVMS archives RTCM messages and sends RTCM message to the Operating System
Data Recorder observes Archived RTCM messages through inspection of the TCVMS Archive Data storage folder and either through inspection of the TCVMS log file (post processing) or through a TCVMS engineering interface in real-time
CVE-TCVMS010-v01
Operational Performance Metrics Configuration
Demonstration Verify Traffic Management Staff can specify a performance metric to the TCVMS (and specify how it is calculated)
NOTE: This is an Operating System Test Case
Traffic Management Center
Test Conductor
Data Producer
Traffic Data Consumer
Data Recorder
Traffic Management Staff enters a performance metric in the TCVMS User Interface
Data Recorder observes Traffic Management Staff entering Performance Metrics into the TCVMS User Interface
Chapter 6. Test Cases
52 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
Test Case ID Test Type Function Verification Method Test Objective
CVE Test Locations Tester Role Metric Pass Criteria
CVE-TCVMS011-v01
First Article Performance Metrics Reporting
Demonstration Verify generated performance metrics are uploaded to the Operating System.
NOTE: This is an Operating System Test Case
Traffic Management Center
Test Conductor
Data Producer
Traffic Data Consumer
Data Recorder
TCVMS sends performance metric to the Operating System
Data Recorder observes the TCVMS sending performance metrics to the Operating System; either through inspection of the TCVMS log file (post processing) or through a TCVMS engineering interface in real-time
CVE-TCVMS012-v01
First Article PII Removal Demonstration Verify PII is removed prior to sending data to Operating System
NOTE: This is an Operating System Test Case
Traffic Management Center
Test Conductor
Data Producer
Traffic Data Consumer
Data Recorder
Data sent to Operating System does NOT contain PII
Data Recorder observes PII has been removed from data prior to send to the Operating System through inspection of a TCVMS log file of data sent to the Operating System
CVE-TCVMS013-v01
First Article RTCM Configuration
Demonstration Verify Traffic Management Staff can specify a source for each RSU to receive RTCM data.
Traffic Management Center
Test Conductor
Data Producer
Traffic Data Consumer
Data Recorder
Traffic Management Staff specify a RTCM source for each RSU
Data Recorder observes Traffic Management Staff specify RTCM source per RSU through the TCVMS User Interface
Chapter 6. Test Cases
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 53
Test Case ID Test Type Function Verification Method Test Objective
CVE Test Locations Tester Role Metric Pass Criteria
CVE-TCVMS014-v01
First Article Unauthorized Network Activity Detection
Demonstration Verify TCVMS can detect unauthorized network activity.
NOTE: This is a Traffic Management Center Test Case
Traffic Management Center
Test Conductor
Data Producer
Traffic Data Consumer
Data Recorder
TCVMS detects unauthorized network activity
Data Recorder observes TCVMS detecting unauthorized network activity; either through inspection of the TCVMS log file (post processing) or through a TCVMS engineering interface in real-time
CVE-TCVMS015-v01
First Article Traffic Signal Cabinet Tamper Detection
Demonstration Verify TCVMS can detect traffic signal cabinet tampering and issue the proper notifications to Traffic CV management staff.
NOTE: This is a Traffic Management Center Test Case
Traffic Management Center
Test Conductor
Data Producer
Traffic Data Consumer
Data Recorder
TCVMS detects Cabinet Tamper Event and provides appropriate notification to Traffic Management Staff through email
Data Recorder observes notification of Cabinet Tamper Event
CVE-TCVMS016-v01
First Article RSU Remote Configuration
Demonstration Verify Traffic Management Staff can modify configurable setting of an RSU using the TCVMS.
Traffic Management Center
Test Conductor
Data Producer
Traffic Data Consumer
Data Recorder
Traffic Management Staff modifies RSU configuration parameters through the TCVMS User Interface
Data Recorder observes Traffic Management Staff updating RSU configuration parameters through the TCVMS User Interface and inspecting the configuration of the RSU(s) directly
Chapter 6. Test Cases
54 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
Test Case ID Test Type Function Verification Method Test Objective
CVE Test Locations Tester Role Metric Pass Criteria
CVE-TCVMS017-v01
First Article Dashboard Demonstration Verify Traffic management Staff have access to the following RSU data via the TCVMS: uptime status, connectivity, RSU graphical display, etc.
Traffic Management Center
Test Conductor
Data Producer
Traffic Data Consumer
Data Recorder
TCVMS User Interface displays RSU data
Data Recorder observes RSU data through the TCVMS User Interface and through inspection of the RSU(s) directly
CVE-TCVMS018-v01
First Article Unauthorized Network Activity Notification
Demonstration Verify alert is issued for unauthorized network activity using Dashboard icon and an email is sent to Traffic CV Management Staff
NOTE: This is a Traffic Management Center Test Case
Traffic Management Center
Test Conductor
Data Producer
Traffic Data Consumer
Data Recorder
TCVMS detects unauthorized network activity and provides appropriate notification to Traffic Management Staff through a Dashboard icon and email
Data Recorder observes notification of unauthorized network activity
CVE-TCVMS019-v01
First Article RSU Status Notification
Demonstration Verify alert is issued when RSU is not operating as intended (degraded/failure conditions, or limited network connectivity) using Dashboard icon and an email is to Traffic CV Management Staff
NOTE: This is a Traffic Management Center Test Case
Traffic Management Center
Test Conductor
Data Producer
Traffic Data Consumer
Data Recorder
TCVMS sends a notification to Traffic Management Staff of unauthorized network activity through a Dashboard icon and e-mail to TCVMS Staff
Data Recorder observes Traffic Management Staff notification
Chapter 6. Test Cases
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 55
Test Case ID Test Type Function Verification Method Test Objective
CVE Test Locations Tester Role Metric Pass Criteria
CVE-TCVMS020-v01
First Article Log notification and emails sent to staff
Demonstration Verify TCVMS logs all alerts/emails to Traffic CV Management Staff
NOTE: This is a Traffic Management Center Test Case
Traffic Management Center
Test Conductor
Data Producer
Traffic Data Consumer
Data Recorder
TCVMS logs alerts and e-mails sent to TCVMS Staff
Data Recorder observes TCVMS log files containing alerts and e-mail notifications
CVE-TCVMS021-v01
First Article Archived Data Query
Demonstration Verify Traffic Management Staff can query archived data on the TCVMS
NOTE: This is an Operating System Test Case
Traffic Management Center
Test Conductor
Data Producer
Traffic Data Consumer
Data Recorder
Traffic Management Staff enters data query parameters into the TCVMS User Interface
Data Recorder observes Traffic Management Staff entering data query parameters into the TCVMS User Interface
Source: City of Columbus
6.7. TRANSIT CV MANAGEMENT SYSTEM
Table 11: Transit CV Management System Test Cases
Test Case ID Test Type Function Verification Method Test Objective
CVE Test Locations Tester Role Metric Pass Criteria
CVE-TrCVMS001-v01 First Article Performance Metrics Configuration
Demonstration Verify Transit Management Staff can specify a performance metric to the TrCVMS (and specify how it is calculated)
COTA McKinley Ave Garage, COTA Fields Ave Garage
Test Conductor
Data Producer
Transit Data Consumer
Data Recorder
Transit Management Staff enters a performance metric in the TrCVMS User Interface
Data Recorder observes Transit Management Staff entering Performance Metrics into the TrCVMS User Interface
Chapter 6. Test Cases
56 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
Test Case ID Test Type Function Verification Method Test Objective
CVE Test Locations Tester Role Metric Pass Criteria
CVE-TrCVMS002-v01 First Article Performance Metrics Reporting
Demonstration Verify generated performance metrics are uploaded to the Smart Columbus Operating System.
COTA McKinley Ave Garage, COTA Fields Ave Garage
Test Conductor
Data Producer
Transit Data Consumer
Data Recorder
TrCVMS sends performance metric to the Operating System
Data Recorder observes the TrCVMS sending performance metrics to the Operating System; either through inspection of the TrCVMS log file (post processing) or through a TrCVMS engineering interface in real-time
CVE-TrCVMS003-v01 First Article Archived Data Query
Demonstration Verify Transit Management Staff can query archived data on the TrCVMS
COTA McKinley Ave Garage, COTA Fields Ave Garage
Test Conductor
Data Producer
Transit Data Consumer
Data Recorder
Transit Management Staff enters data query parameters into the TrCVMS User Interface
Data Recorder observes Transit Management Staff entering data query parameters into the TrCVMS User Interface
CVE-TrCVMS004-v01 First Article Receive/Archive Transit Vehicle Interaction Event
Demonstration Verify the TrCVMS archives the Transit Vehicle Interaction Events and forwards them to the Operating System
COTA McKinley Ave Garage, COTA Fields Ave Garage
Test Conductor
Data Producer
Transit Data Consumer
Data Recorder
TrCVMS archives data from Transit Vehicle logs
Data Recorder observes Archived Transit Vehicle log files through inspection of the TrCVMS system Archive Data storage folder
Source: City of Columbus
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 57
Chapter 7. Test Scenarios
Test scenarios are critical to the acceptance of the CVE, as it tests the system holistically, end-to-end, from all active participant viewpoints. Scenarios are made up of a series of test procedures used
to simulate the system in a real-world operational environment. This approach validates the system’s ability to meet the concepts established through the project ConOps and forms the basis of
acceptance testing for the given production release.
The CVE capability and performance will be deemed acceptable provided all testing elements within a scenario successfully passes the test from each participant’s viewpoint.
The scenarios outlined below are reflective of the capabilities the system must be able to perform to receive acceptance of the CVE. The CVE capability and performance will be deemed acceptable
provided all testing elements within a scenario successfully passes the test from each participant’s viewpoint.
Table 12 provides the comprehensive list of scenarios that will be tested with traceability to the requirement(s) published in the SyRS, Interface Control Document (ICD) and ConOps.
Table 12: Testing Scenarios with Requirements Traceability
Function Acceptance Test Scenario Scenario Description
Test Case ID Reference
Requirement ID Reference
Emergency Electronic Brake Light
CVE-ATS0100-v01 LDV driver receives appropriate EEBL alerts
CVE-OBU001-v01
CVE-OBU003-v01
CVE-App001-v01
CVE-App008-v01
CVE-RSU007-v01
CVE-RSU012-v01
CVE-MH008-v01
CVE-DR3005-V01
CVE-DR3006-V01
CVE-PR1105-V01
CVE-PR3003-V01
CVE-PR3007-V01
CVE-PR3008-V01
CVE-PR3009-V01
CVE-DR1374-V01
CVE-DR1375-V01
CVE-FN1319-V01
CVE-FN1321-V01
CVE-FN3111-V01
CVE-FN3112-V01
CVE-FN3227-V01
CVE-FN3228-V01
Forward Collision Warning
CVE-ATS0101-v01 LDV driver receives appropriate FCW alerts
CVE-OBU001-v01
CVE-OBU002-v01
CVE-App002-v01
CVE-App008-v01
CVE-RSU007-v01
CVE-RSU012-v01
CVE-MH008-v01
CVE-DR3005-V01
CVE-DR3006-V01
CVE-PR1105-V01
CVE-PR3003-V01
CVE-PR3007-V01
CVE-PR3008-V01
CVE-PR3009-V01
CVE-DR1374-V01
CVE-DR1375-V01
CVE-FN1319-V01
CVE-FN1321-V01
CVE-FN3111-V01
CVE-FN3112-V01
CVE-FN3227-V01
CVE-FN3228-V01
Intersection Movement Assist
CVE-ATS0102-v01 LDV driver receives appropriate IMA alerts
CVE-OBU001-v01
CVE-OBU004-v01
CVE-App004-v01
CVE-DR3005-V01
CVE-DR3006-V01
CVE-PR1105-V01
CVE-PR3007-V01
CVE-PR3008-V01
CVE-PR3009-V01
CVE-DR1375-V01
CVE-FN1319-V01
CVE-FN1321-V01
CVE-FN3112-V01
CVE-FN3227-V01
CVE-FN3228-V01
Chapter 7. Test Scenarios
58 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
Function Acceptance Test Scenario Scenario Description
Test Case ID Reference
Requirement ID Reference
CVE-App008-v01
CVE-RSU007-v01
CVE-RSU012-v01
CVE-MH008-v01
CVE-PR3003-V01 CVE-DR1374-V01 CVE-FN3111-V01
Lane Change Warning/Blind Spot Warning
CVE-ATS0103-v01 LDV driver receives appropriate LCW alerts
CVE-OBU001-v01
CVE-OBU005-v01
CVE-App003-v01
CVE-App008-v01
CVE-RSU007-v01
CVE-RSU012-v01
CVE-MH008-v01
CVE-DR3005-V01
CVE-DR3006-V01
CVE-PR1105-V01
CVE-PR3003-V01
CVE-PR3007-V01
CVE-PR3008-V01
CVE-PR3009-V01
CVE-DR1374-V01
CVE-DR1375-V01
CVE-FN1319-V01
CVE-FN1321-V01
CVE-FN3111-V01
CVE-FN3112-V01
CVE-FN3227-V01
CVE-FN3228-V01
Traffic Signal Priority/Preemption
CVE-ATS0104-v01 Transit vehicles receive appropriate Traffic Signal Priority
CVE-OBU014-v01
CVE-RSU002-v01
CVE-RSU003-v01
CVE-RSU004-v01
CVE-RSU005-v01
CVE-RSU006-v01
CVE-RSU007-v01
CVE-RSU008-v01
CVE-RSU011-v01
CVE-RSU012-v01
CVE-MH002-v01
CVE-MH003-v01
CVE-MH004-v01
CVE-MH006-v01
CVE-MH008-v01
CVE-TSC001-v01
CVE-TSC002-v01
CVE-TSC003-v01
CVE-TSC004-v01
CVE-DR1144-V01
CVE-DR1145-V01
CVE-DR1146-V01
CVE-DR1147-V01
CVE-DR1148-V01
CVE-DR1149-V01
CVE-DR1150-V01
CVE-DR1151-V01
CVE-DR1152-V01
CVE-DR1153-V01
CVE-DR1154-V01
CVE-DR1155-V01
CVE-DR1156-V01
CVE-DR1157-V01
CVE-DR1158-V01
CVE-DR1159-V01
CVE-DR1160-V01
CVE-DR1161-V01
CVE-DR1162-V01
CVE-DR1163-V01
CVE-DR1164-V01
CVE-DR1165-V01
CVE-DR1166-V01
CVE-DR1378-V01
CVE-DR1379-V01
CVE-DR1380-V01
CVE-DR1381-V01
CVE-DR1382-V01
CVE-DR1383-V01
CVE-DR1384-V01
CVE-DR1385-V01
CVE-DR1386-V01
CVE-DR1387-V01
CVE-DR1388-V01
CVE-DR1389-V01
CVE-DR1390-V01
CVE-DR1391-V01
CVE-DR1392-V01
CVE-DR1393-V01
CVE-DR1394-V01
CVE-DR1395-V01
CVE-DR1396-V01
CVE-DR1397-V01
CVE-DR1398-V01
CVE-PR1399-V01
CVE-PR1400-V01
CVE-DR1422-V01
CVE-DR1423-V01
CVE-DR1424-V01
CVE-DR1425-V01
CVE-DR1426-V01
CVE-DR1427-V01
CVE-DR1428-V01
CVE-DR1429-V01
CVE-DR1430-V01
CVE-DR1431-V01
CVE-DR1432-V01
CVE-DR1433-V01
CVE-DR1434-V01
CVE-DR1435-V01
CVE-DR1436-V01
CVE-PR2999-V01
CVE-AR3121-V01
CVE-AR3122-V01
CVE-FN1113-V01
CVE-FN1308-V01
CVE-FN1309-V01
CVE-FN1311-V01
CVE-FN1312-V01
CVE-FN3238-V01
CVE-FN3240-V01
CVE-IF1340-V01
CVE-IF1342-V01
CVE-IF1345-V01
CVE-IF1346-V01
CVE-IF1347-V01
CVE-IF1361-V01
CVE-IF1363-V01
CVE-IF2985-V01
CVE-PR1365-V01
CVE-PR1366-V01
CVE-PR1369-V01
CVE-PR3213-V01
CVE-PR3226-V01
CVE-PR3237-V01
CVE-SR3123-V01
CVE-SR3125-V01
CVE-SR3127-V01
CVE-SR3130-V01
CVE-FN1439-V01
CVE-FN1441-V01
CVE-FN1443-V01
Chapter 7. Test Scenarios
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 59
Function Acceptance Test Scenario Scenario Description
Test Case ID Reference
Requirement ID Reference
CVE-DR1167-V01
CVE-DR1168-V01
CVE-DR1169-V01
CVE-DR1170-V01
CVE-DR1171-V01
CVE-DR1172-V01
CVE-DR1173-V01
CVE-DR1174-V01
CVE-DR1175-V01
CVE-DR1176-V01
CVE-DR1177-V01
CVE-DR1178-V01
CVE-DR1179-V01
CVE-DR1181-V01
CVE-DR1182-V01
CVE-PR1183-V01
CVE-PR2993-V01
CVE-DR1374-V01
CVE-DR1375-V01
CVE-PR1401-V01
CVE-DR1402-V01
CVE-DR1404-V01
CVE-DR1405-V01
CVE-DR1406-V01
CVE-DR1407-V01
CVE-DR1408-V01
CVE-DR1409-V01
CVE-DR1410-V01
CVE-DR1411-V01
CVE-DR1412-V01
CVE-DR1413-V01
CVE-DR1414-V01
CVE-DR1415-V01
CVE-DR1416-V01
CVE-DR1417-V01
CVE-DR1418-V01
CVE-PR2995-V01
CVE-DR1420-V01
CVE-FN1313-V01
CVE-FN1314-V01
CVE-FN1319-V01
CVE-FN1321-V01
CVE-FN1325-V01
CVE-FN1327-V01
CVE-FN1333-V01
CVE-FN1335-V01
CVE-FN2974-V01
CVE-FN2976-V01
CVE-FN2980-V01
CVE-FN2983-V01
CVE-FN2990-V01
CVE-FN3000-V01
CVE-FN3010-V01
CVE-FN3111-V01
CVE-FN3112-V01
CVE-FN3227-V01
CVE-FN3228-V01
CVE-FN3236-V01
CVE-FN1448-V01
CVE-FN3002-V01
CVE-DR1477-V01
CVE-DR1478-V01
CVE-FN1498-V01
CVE-FN1499-V01
CVE-FN1503-V01
CVE-FN1504-V01
CVE-FN1505-V01
CVE-FN1509-V01
CVE-FN1510-V01
CVE-FN1511-V01
CVE-FN1512-V01
CVE-FN1513-V01
CVE-FN1514-V01
CVE-FN1518-V01
CVE-FN1519-V01
CVE-FN1501-V01
CVE-PR1529-V01
CVE-PR1530-V01
Traffic Signal Priority/Preemption
CVE-ATS0105-v01 EVs receive appropriate Traffic Signal Preemption
CVE-OBU012-v01
CVE-OBU013-v01
CVE-App007-v01
CVE-App008-v01
CVE-RSU002-v01
CVE-RSU003-v01
CVE-RSU004-v01
CVE-RSU005-v01
CVE-RSU006-v01
CVE-RSU007-v01
CVE-RSU008-v01
CVE-RSU011-v01
CVE-RSU012-v01
CVE-MH002-v01
CVE-DR1144-V01
CVE-DR1145-V01
CVE-DR1146-V01
CVE-DR1147-V01
CVE-DR1148-V01
CVE-DR1149-V01
CVE-DR1150-V01
CVE-DR1151-V01
CVE-DR1152-V01
CVE-DR1153-V01
CVE-DR1154-V01
CVE-DR1155-V01
CVE-DR1156-V01
CVE-DR1157-V01
CVE-DR1378-V01
CVE-DR1379-V01
CVE-DR1380-V01
CVE-DR1381-V01
CVE-DR1382-V01
CVE-DR1383-V01
CVE-DR1384-V01
CVE-DR1385-V01
CVE-DR1386-V01
CVE-DR1387-V01
CVE-DR1388-V01
CVE-DR1389-V01
CVE-DR1390-V01
CVE-DR1391-V01
CVE-DR1422-V01
CVE-DR1423-V01
CVE-DR1424-V01
CVE-DR1425-V01
CVE-DR1426-V01
CVE-DR1427-V01
CVE-DR1428-V01
CVE-DR1429-V01
CVE-DR1430-V01
CVE-DR1431-V01
CVE-DR1432-V01
CVE-DR1433-V01
CVE-DR1434-V01
CVE-DR1435-V01
CVE-FN3236-V01
CVE-FN3238-V01
CVE-FN3240-V01
CVE-IF1340-V01
CVE-IF1342-V01
CVE-IF1345-V01
CVE-IF1346-V01
CVE-IF1347-V01
CVE-IF1361-V01
CVE-IF1363-V01
CVE-IF2986-V01
CVE-PR1365-V01
CVE-PR1366-V01
CVE-PR1369-V01
Chapter 7. Test Scenarios
60 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
Function Acceptance Test Scenario Scenario Description
Test Case ID Reference
Requirement ID Reference
CVE-MH003-v01
CVE-MH005-v01
CVE-MH006-v01
CVE-MH008-v01
CVE-TSC001-v01
CVE-TSC002-v01
CVE-TSC003-v01
CVE-TSC005-v01
CVE-DR1158-V01
CVE-DR1159-V01
CVE-DR1160-V01
CVE-DR1161-V01
CVE-DR1162-V01
CVE-DR1163-V01
CVE-DR1164-V01
CVE-DR1165-V01
CVE-DR1166-V01
CVE-DR1167-V01
CVE-DR1168-V01
CVE-DR1169-V01
CVE-DR1170-V01
CVE-DR1171-V01
CVE-DR1172-V01
CVE-DR1173-V01
CVE-DR1174-V01
CVE-DR1175-V01
CVE-DR1176-V01
CVE-DR1177-V01
CVE-DR1178-V01
CVE-DR1179-V01
CVE-DR1181-V01
CVE-DR1182-V01
CVE-PR1183-V01
CVE-PR2993-V01
CVE-DR1374-V01
CVE-DR1375-V01
CVE-DR1392-V01
CVE-DR1393-V01
CVE-DR1394-V01
CVE-DR1395-V01
CVE-DR1396-V01
CVE-DR1397-V01
CVE-DR1398-V01
CVE-PR1399-V01
CVE-PR1400-V01
CVE-PR1401-V01
CVE-DR1402-V01
CVE-DR1404-V01
CVE-DR1405-V01
CVE-DR1406-V01
CVE-DR1407-V01
CVE-DR1408-V01
CVE-DR1409-V01
CVE-DR1410-V01
CVE-DR1411-V01
CVE-DR1412-V01
CVE-DR1413-V01
CVE-DR1414-V01
CVE-DR1415-V01
CVE-DR1416-V01
CVE-DR1417-V01
CVE-DR1418-V01
CVE-PR2995-V01
CVE-DR1420-V01
CVE-DR1436-V01
CVE-PR2999-V01
CVE-AR3121-V01
CVE-AR3122-V01
CVE-FN1113-V01
CVE-FN1308-V01
CVE-FN1309-V01
CVE-FN1311-V01
CVE-FN1312-V01
CVE-FN1313-V01
CVE-FN1314-V01
CVE-FN1319-V01
CVE-FN1321-V01
CVE-FN1325-V01
CVE-FN1327-V01
CVE-FN1333-V01
CVE-FN1335-V01
CVE-FN2975-V01
CVE-FN2977-V01
CVE-FN2981-V01
CVE-FN2991-V01
CVE-FN3000-V01
CVE-FN3010-V01
CVE-FN3109-V01
CVE-FN3111-V01
CVE-FN3112-V01
CVE-FN3227-V01
CVE-FN3228-V01
CVE-PR3213-V01
CVE-PR3226-V01
CVE-PR3237-V01
CVE-SR3123-V01
CVE-SR3125-V01
CVE-SR3127-V01
CVE-SR3130-V01
CVE-FN1439-V01
CVE-FN1441-V01
CVE-FN1443-V01
CVE-FN1448-V01
CVE-FN3002-V01
CVE-FN1497-V01
CVE-FN1500-V01
CVE-FN3107-V01
CVE-PR1531-V01
CVE-FN1498-V01
CVE-FN1499-V01
CVE-FN1503-V01
CVE-FN1504-V01
CVE-FN1505-V01
CVE-FN1513-V01
CVE-FN1514-V01
CVE-FN1515-V01
CVE-FN1516-V01
CVE-FN1517-V01
CVE-FN1519-V01
Traffic Signal Priority/Preemption
CVE-ATS0106-v01 Heavy Duty vehicles receive appropriate Traffic Signal Priority
CVE-OBU016-v01
CVE-OBU017-v01
CVE-App008-v01
CVE-RSU002-v01
CVE-RSU003-v01
CVE-RSU004-v01
CVE-PR3007-V01
CVE-PR3008-V01
CVE-PR3009-V01
CVE-DR1144-V01
CVE-DR1145-V01
CVE-DR1146-V01
CVE-DR1175-V01
CVE-DR1176-V01
CVE-DR1177-V01
CVE-DR1178-V01
CVE-DR1179-V01
CVE-DR1181-V01
CVE-PR1401-V01
CVE-AR3121-V01
CVE-AR3122-V01
CVE-FN1113-V01
CVE-FN1308-V01
CVE-FN1309-V01
CVE-IF1361-V01
CVE-IF1363-V01
CVE-PR1365-V01
CVE-PR1366-V01
CVE-PR1369-V01
CVE-PR3213-V01
Chapter 7. Test Scenarios
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 61
Function Acceptance Test Scenario Scenario Description
Test Case ID Reference
Requirement ID Reference
CVE-RSU005-v01
CVE-RSU006-v01
CVE-RSU007-v01
CVE-RSU008-v01
CVE-RSU011-v01
CVE-RSU012-v01
CVE-MH002-v01
CVE-MH003-v01
CVE-MH004-v01
CVE-MH006-v01
CVE-MH008-v01
CVE-TSC001-v01
CVE-TSC002-v01
CVE-TSC003-v01
CVE-TSC004-v01
CVE-DR1147-V01
CVE-DR1148-V01
CVE-DR1149-V01
CVE-DR1150-V01
CVE-DR1151-V01
CVE-DR1152-V01
CVE-DR1153-V01
CVE-DR1154-V01
CVE-DR1155-V01
CVE-DR1156-V01
CVE-DR1157-V01
CVE-DR1158-V01
CVE-DR1159-V01
CVE-DR1160-V01
CVE-DR1161-V01
CVE-DR1162-V01
CVE-DR1163-V01
CVE-DR1164-V01
CVE-DR1165-V01
CVE-DR1166-V01
CVE-DR1167-V01
CVE-DR1168-V01
CVE-DR1169-V01
CVE-DR1170-V01
CVE-DR1171-V01
CVE-DR1172-V01
CVE-DR1173-V01
CVE-DR1174-V01
CVE-DR1182-V01
CVE-PR1183-V01
CVE-PR2993-V01
CVE-DR1374-V01
CVE-DR1375-V01
CVE-DR1378-V01
CVE-DR1379-V01
CVE-DR1380-V01
CVE-DR1381-V01
CVE-DR1382-V01
CVE-DR1383-V01
CVE-DR1384-V01
CVE-DR1385-V01
CVE-DR1386-V01
CVE-DR1387-V01
CVE-DR1388-V01
CVE-DR1389-V01
CVE-DR1390-V01
CVE-DR1391-V01
CVE-DR1392-V01
CVE-DR1393-V01
CVE-DR1394-V01
CVE-DR1395-V01
CVE-DR1396-V01
CVE-DR1397-V01
CVE-DR1398-V01
CVE-PR1399-V01
CVE-PR1400-V01
CVE-FN1311-V01
CVE-FN1312-V01
CVE-FN1313-V01
CVE-FN1319-V01
CVE-FN1321-V01
CVE-FN1325-V01
CVE-FN1327-V01
CVE-FN1333-V01
CVE-FN1335-V01
CVE-FN2973-V01
CVE-FN2979-V01
CVE-FN2982-V01
CVE-FN2987-V01
CVE-FN2988-V01
CVE-FN2989-V01
CVE-FN3111-V01
CVE-FN3112-V01
CVE-FN3227-V01
CVE-FN3228-V01
CVE-FN3236-V01
CVE-FN3238-V01
CVE-FN3240-V01
CVE-IF1340-V01
CVE-IF1342-V01
CVE-IF1345-V01
CVE-IF1346-V01
CVE-IF1347-V01
CVE-IF1359-V01
CVE-PR3226-V01
CVE-PR3237-V01
CVE-SR3123-V01
CVE-SR3125-V01
CVE-SR3127-V01
CVE-SR3130-V01
CVE-FN1439-V01
CVE-FN1441-V01
CVE-FN1443-V01
CVE-FN1448-V01
CVE-FN3002-V01
CVE-FN1502-V01
CVE-PR1527-V01
CVE-PR1528-V01
CVE-FN1498-V01
CVE-FN1499-V01
CVE-FN1503-V01
CVE-FN1504-V01
CVE-FN1505-V01
CVE-FN1509-V01
CVE-FN1510-V01
CVE-FN1511-V01
CVE-FN1512-V01
CVE-FN1513-V01
CVE-FN1514-V01
CVE-FN1518-V01
CVE-FN1519-V01
Vehicle Data for Traffic Operations
CVE-ATS0107-v01 TCVMS and Operating System receive Vehicle Data from OBUs and RSUs
CVE-OBU001-v01
CVE-RSU001-v01
CVE-MH007-v01
CVE-TCVMS001-v01
CVE-TCVMS002-v01
CVE-TCVMS003-v01
CVE-DR3005-V01
CVE-DR3006-V01
CVE-PR1105-V01
CVE-PR3003-V01
CVE-PR3007-V01
CVE-PR3008-V01
CVE-FN1335-V01
CVE-FN2987-V01
CVE-FN2988-V01
CVE-FN2989-V01
CVE-FN3233-V01
CVE-FN3241-V01
CVE-SR3128-V01
CVE-SR3129-V01
CVE-SR3242-V01
CVE-DR1276-V01
CVE-FN1437-V01
CVE-FN1438-V01
CVE-FN1456-V01
CVE-FN1463-V01
CVE-FN2909-V01
CVE-FN2911-V01
CVE-FN3002-V01
CVE-FN3030-V01
Chapter 7. Test Scenarios
62 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
Function Acceptance Test Scenario Scenario Description
Test Case ID Reference
Requirement ID Reference
CVE-TCVMS004-v01
CVE-TCVMS005-v01
CVE-TCVMS006-v01
CVE-TCVMS007-v01
CVE-TCVMS008-v01
CVE-TCVMS009-v01
CVE-TCVMS010-v01
CVE-TCVMS011-v01
CVE-TCVMS012-v01
CVE-TCVMS013-v01
CVE-TCVMS014-v01
CVE-TCVMS015-v01
CVE-TCVMS016-v01
CVE-TCVMS017-v01
CVE-TCVMS018-v01
CVE-TCVMS019-v01
CVE-TCVMS020-v01
CVE-TCVMS021-v01
CVE-PR3009-V01
CVE-AR3121-V01
CVE-AR3122-V01
CVE-FN1113-V01
CVE-FN1308-V01
CVE-FN1309-V01
CVE-FN1311-V01
CVE-FN1312-V01
CVE-FN1313-V01
CVE-FN1317-V01
CVE-FN1318-V01
CVE-FN1322-V01
CVE-FN1325-V01
CVE-FN1327-V01
CVE-FN1328-V01
CVE-FN1333-V01
CVE-IF1361-V01
CVE-IF1362-V01
CVE-IF1363-V01
CVE-PR1365-V01
CVE-PR1366-V01
CVE-PR1369-V01
CVE-PR3229-V01
CVE-PR3243-V01
CVE-PR3244-V01
CVE-PY2912-V01
CVE-PY3120-V01
CVE-SR1373-V01
CVE-SR3123-V01
CVE-SR3124-V01
CVE-SR3126-V01
CVE-FN1439-V01
CVE-FN1440-V01
CVE-FN1441-V01
CVE-FN1442-V01
CVE-FN1443-V01
CVE-FN1444-V01
CVE-FN1445-V01
CVE-FN1446-V01
CVE-FN1447-V01
CVE-FN1448-V01
CVE-FN1449-V01
CVE-FN1451-V01
CVE-FN1452-V01
CVE-FN1453-V01
CVE-FN1454-V01
CVE-FN3032-V01
CVE-FN3041-V01
CVE-FN3110-V01
CVE-PR3029-V01
CVE-PR3031-V01
CVE-PR3033-V01
CVE-PY3034-V01
CVE-DR1562-V01
CVE-FN1585-V01
CVE-FN1586-V01
CVE-FN1587-V01
CVE-FN1588-V01
CVE-FN1589-V01
CVE-FN1590-V01
CVE-FN1591-V01
Transit Vehicle Interaction Event Recording
CVE-ATS0108-v01 Transit Vehicles record appropriate interaction events
CVE-OBU002-v01
CVE-OBU003-v01
CVE-OBU004-v01
CVE-OBU005-v01
CVE-OBU006-v01
CVE-OBU007-v01
CVE-OBU008-v01
CVE-OBU009-v01
CVE-OBU010-v01
CVE-OBU015-v01
CVE-RSU007-v01
CVE-RSU012-v01
CVE-MH008-v01
CVE-DR1144-V01
CVE-DR1145-V01
CVE-DR1146-V01
CVE-DR1147-V01
CVE-DR1148-V01
CVE-DR1149-V01
CVE-DR1150-V01
CVE-DR1151-V01
CVE-DR1152-V01
CVE-DR1153-V01
CVE-DR1154-V01
CVE-DR1155-V01
CVE-DR1156-V01
CVE-DR1157-V01
CVE-DR1158-V01
CVE-DR1159-V01
CVE-DR1168-V01
CVE-DR1169-V01
CVE-DR1170-V01
CVE-DR1171-V01
CVE-DR1172-V01
CVE-DR1173-V01
CVE-DR1174-V01
CVE-DR1175-V01
CVE-DR1176-V01
CVE-DR1177-V01
CVE-DR1178-V01
CVE-DR1179-V01
CVE-DR1181-V01
CVE-DR1182-V01
CVE-PR1183-V01
CVE-PR2993-V01
CVE-DR1412-V01
CVE-DR1413-V01
CVE-DR1414-V01
CVE-DR1415-V01
CVE-DR1416-V01
CVE-DR1417-V01
CVE-DR1418-V01
CVE-PR2995-V01
CVE-DR1420-V01
CVE-DR1422-V01
CVE-DR1423-V01
CVE-DR1424-V01
CVE-DR1425-V01
CVE-DR1426-V01
CVE-DR1427-V01
CVE-DR1428-V01
CVE-PR2999-V01
CVE-AR3121-V01
CVE-AR3122-V01
CVE-FN1113-V01
CVE-FN1308-V01
CVE-FN1309-V01
CVE-FN1311-V01
CVE-FN1312-V01
CVE-FN1313-V01
CVE-FN1314-V01
CVE-FN2972-V01
CVE-FN2974-V01
CVE-FN2976-V01
CVE-FN2980-V01
CVE-FN2983-V01
CVE-FN3000-V01
Chapter 7. Test Scenarios
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 63
Function Acceptance Test Scenario Scenario Description
Test Case ID Reference
Requirement ID Reference
CVE-DR1160-V01
CVE-DR1161-V01
CVE-DR1162-V01
CVE-DR1163-V01
CVE-DR1164-V01
CVE-DR1165-V01
CVE-DR1166-V01
CVE-DR1167-V01
CVE-DR1374-V01
CVE-DR1375-V01
CVE-DR1406-V01
CVE-DR1407-V01
CVE-DR1408-V01
CVE-DR1409-V01
CVE-DR1410-V01
CVE-DR1411-V01
CVE-DR1429-V01
CVE-DR1430-V01
CVE-DR1431-V01
CVE-DR1432-V01
CVE-DR1433-V01
CVE-DR1434-V01
CVE-DR1435-V01
CVE-DR1436-V01
CVE-FN3010-V01
CVE-FN3111-V01
CVE-FN3112-V01
CVE-FN3227-V01
CVE-FN3228-V01
CVE-IF2978-V01
CVE-IF2985-V01
Red Light Violation Warning
CVE-ATS0109-v01 LDV driver receives appropriate RLVW alerts
CVE-OBU006-v01
CVE-App005-v01
CVE-App008-v01
CVE-RSU006-v01
CVE-RSU007-v01
CVE-RSU011-v01
CVE-RSU012-v01
CVE-MH001-v01
CVE-MH008-v01
CVE-TSC001-v01
CVE-DR1144-V01
CVE-DR1145-V01
CVE-DR1146-V01
CVE-DR1147-V01
CVE-DR1148-V01
CVE-DR1149-V01
CVE-DR1150-V01
CVE-DR1151-V01
CVE-DR1152-V01
CVE-DR1153-V01
CVE-DR1154-V01
CVE-DR1155-V01
CVE-DR1156-V01
CVE-DR1157-V01
CVE-DR1158-V01
CVE-DR1159-V01
CVE-DR1160-V01
CVE-DR1161-V01
CVE-DR1162-V01
CVE-DR1163-V01
CVE-DR1164-V01
CVE-DR1165-V01
CVE-DR1166-V01
CVE-DR1167-V01
CVE-DR1168-V01
CVE-DR1169-V01
CVE-DR1171-V01
CVE-DR1172-V01
CVE-DR1173-V01
CVE-DR1174-V01
CVE-DR1175-V01
CVE-DR1176-V01
CVE-DR1177-V01
CVE-DR1178-V01
CVE-DR1179-V01
CVE-DR1181-V01
CVE-DR1182-V01
CVE-PR1183-V01
CVE-PR2993-V01
CVE-DR1374-V01
CVE-DR1375-V01
CVE-DR1378-V01
CVE-DR1379-V01
CVE-DR1380-V01
CVE-DR1381-V01
CVE-DR1382-V01
CVE-DR1383-V01
CVE-DR1384-V01
CVE-DR1385-V01
CVE-DR1386-V01
CVE-DR1387-V01
CVE-DR1388-V01
CVE-DR1390-V01
CVE-DR1391-V01
CVE-DR1392-V01
CVE-DR1393-V01
CVE-DR1394-V01
CVE-DR1395-V01
CVE-DR1396-V01
CVE-DR1397-V01
CVE-DR1398-V01
CVE-PR1399-V01
CVE-PR1400-V01
CVE-PR1401-V01
CVE-AR3121-V01
CVE-AR3122-V01
CVE-FN1113-V01
CVE-FN1308-V01
CVE-FN1309-V01
CVE-FN1311-V01
CVE-FN1312-V01
CVE-FN1313-V01
CVE-FN1319-V01
CVE-FN1321-V01
CVE-FN1325-V01
CVE-FN1327-V01
CVE-FN1333-V01
CVE-FN1335-V01
CVE-FN3112-V01
CVE-FN3227-V01
CVE-FN3228-V01
CVE-FN3236-V01
CVE-FN3238-V01
CVE-FN3240-V01
CVE-IF1342-V01
CVE-IF1345-V01
CVE-IF1346-V01
CVE-IF1353-V01
CVE-IF1356-V01
CVE-IF1357-V01
CVE-IF1358-V01
CVE-PR1365-V01
CVE-PR1366-V01
CVE-PR1369-V01
CVE-PR3213-V01
CVE-PR3226-V01
CVE-PR3237-V01
CVE-SR3125-V01
CVE-SR3127-V01
CVE-SR3130-V01
CVE-FN1439-V01
CVE-FN1441-V01
CVE-FN1443-V01
CVE-FN1448-V01
Chapter 7. Test Scenarios
64 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
Function Acceptance Test Scenario Scenario Description
Test Case ID Reference
Requirement ID Reference
CVE-DR1170-V01 CVE-DR1389-V01 CVE-FN3111-V01 CVE-FN3002-V01
Reduced Speed School Zone
CVE-ATS0110-v01 LDV driver receives appropriate RSSZ alerts
CVE-OBU007-v01
CVE-App006-v01
CVE-App008-v01
CVE-RSU006-v01
CVE-RSU007-v01
CVE-RSU009-v01
CVE-RSU011-v01
CVE-RSU012-v01
CVE-MH008-v01
CVE-DR1144-V01
CVE-DR1145-V01
CVE-DR1146-V01
CVE-DR1147-V01
CVE-DR1148-V01
CVE-DR1149-V01
CVE-DR1150-V01
CVE-DR1151-V01
CVE-DR1152-V01
CVE-DR1153-V01
CVE-DR1154-V01
CVE-DR1155-V01
CVE-DR1156-V01
CVE-DR1157-V01
CVE-DR1158-V01
CVE-DR1159-V01
CVE-DR1160-V01
CVE-DR1161-V01
CVE-DR1162-V01
CVE-DR1163-V01
CVE-DR1164-V01
CVE-DR1165-V01
CVE-DR1166-V01
CVE-DR1167-V01
CVE-DR1168-V01
CVE-DR1169-V01
CVE-DR1170-V01
CVE-DR1171-V01
CVE-DR1172-V01
CVE-DR1173-V01
CVE-DR1174-V01
CVE-DR1175-V01
CVE-DR1176-V01
CVE-DR1177-V01
CVE-DR1178-V01
CVE-DR1179-V01
CVE-DR1181-V01
CVE-DR1182-V01
CVE-PR1183-V01
CVE-PR2993-V01
CVE-DR1374-V01
CVE-DR1375-V01
CVE-DR1292-V01
CVE-DR1294-V01
CVE-DR1296-V01
CVE-DR3089-V01
CVE-DR3090-V01
CVE-DR3091-V01
CVE-DR3092-V01
CVE-DR3093-V01
CVE-AR3121-V01
CVE-AR3122-V01
CVE-FN1113-V01
CVE-FN1308-V01
CVE-FN1309-V01
CVE-FN1310-V01
CVE-FN1311-V01
CVE-FN1313-V01
CVE-FN1316-V01
CVE-FN1319-V01
CVE-FN1321-V01
CVE-FN1325-V01
CVE-FN1327-V01
CVE-FN1333-V01
CVE-FN1335-V01
CVE-FN2972-V01
CVE-FN3111-V01
CVE-FN3112-V01
CVE-FN3227-V01
CVE-FN3228-V01
CVE-FN3236-V01
CVE-FN3238-V01
CVE-FN3240-V01
CVE-IF1341-V01
CVE-IF1342-V01
CVE-IF1353-V01
CVE-IF1360-V01
CVE-IF2978-V01
CVE-PR2994-V01
CVE-SR3125-V01
CVE-SR3127-V01
CVE-SR3130-V01
CVE-FN1438-V01
CVE-FN1439-V01
CVE-FN1440-V01
CVE-FN1441-V01
CVE-FN1442-V01
CVE-FN1443-V01
CVE-FN1447-V01
CVE-FN1448-V01
CVE-FN3001-V01
CVE-FN3002-V01
Vehicle Data for Transit Operations
CVE-ATS0111-v01 TrCVMS receive Vehicle Data from Transit Vehicle OBUs
CVE-OBU009-v01
CVE-OBU010-v01
CVE-OBU011-v01
CVE-TrCVMS001-v01
CVE-TrCVMS003-v01
CVE-TrCVMS004-v01
CVE-TrCVMS006-v01
CVE-AR3121-V01
CVE-AR3122-V01
CVE-FN1113-V01
CVE-FN1308-V01
CVE-FN1309-V01
CVE-FN1311-V01
CVE-FN1312-V01
CVE-FN1313-V01
Source: City of Columbus
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 65
Chapter 8. Test Procedures
8.1. ONBOARD UNIT
Table 13: Onboard Unit Test Procedures
Test Case ID Objective Entrance Criteria Procedure Exit Criteria Data Output
CVE-OBU001-v01 Verify OBU’s broadcast valid BSMs 10 times per second.
No DSRC devices are broadcasting in Test Area
OBU is configured for "normal" operation, including broadcasting BSMs
CVTT is configured to log pcap files on Channel 172
Power on the OBU under test
Power on the Connected Vehicle Test Tool (CVTT)
Capture 5 minutes of BSMs on the CVTT
Review pcap file to confirm OBU broadcast 10 BSMs/sec
OBU broadcasts 10 BSMs/sec pcap file containing BSMs
CVE-OBU002-v01 Verify a host Transit Vehicle OBU logs an FCW-based transit vehicle interaction event only when an FCW threat exists
OBU1 is configured for "normal" operation, including broadcasting BSMs
OBU2 (Transit) is configured for "normal" operation, including broadcasting BSMs and logging FCW events
A location is set up as a typical roadway segment in the Demonstration Area
Vehicle with OBU1 (V1) is stopped in the roadway
Vehicle with OBU2 (V2) (Transit) approaches V1 from behind
Based on BSMs from OBU1, OBU2 determines a threat exists
Since V2 (Transit) does not contain an HMI, OBU2 only records the FCW event
OBU2 records an FCW event OBU2 log file depicting the recorded FCW event
Chapter 8. Test Procedures
66 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
Test Case ID Objective Entrance Criteria Procedure Exit Criteria Data Output
CVE-OBU003-v01 Verify a host Transit Vehicle OBU logs an EEBL-based interaction event only when EEBL threat exists
OBU1 is configured for "normal" operation, including broadcasting BSMs
OBU2 (Transit) is configured for "normal" operation, including broadcasting BSMs and logging EEBL events
A location is set up as a typical roadway segment in the Demonstration Area
Vehicle with OBU1 (V1) leads vehicle with OBU2 (V2) (Transit) on the roadway
V1 performs a hard-braking maneuver such that an EEBL event flag is set in its BSMs
Based on BSMs from OBU1, OBU2 determines a threat exists
Since V2 (Transit) does not contain and HMI, OBU2 only records the EEBL event
OBU2 records an EEBL event OBU2 log file depicting the recorded EEBL event
CVE-OBU004-v01 Verify a host Transit Vehicle OBU logs an IMA-based transit vehicle interaction event only when an IMA threat exists
OBU1 is configured for "normal" operation, including broadcasting BSMs
OBU2 (Transit) is configured for "normal" operation, including broadcasting BSMs and logging IMA events
A location is set up as a typical two lane (opposite direction of travel) roadway segment in the Demonstration Area
Vehicle with OBU1 (V1) approaches vehicle with OBU2 (V2) (Transit) in roadway segment in a lane to the left of V2
V2 begins to turn left in front of V1
Based on BSMs from OBU1, OBU2 determines a threat exists
Since V2 (Transit) does not contain and HMI, OBU2 only records the IMA event
OBU2 records an IMA event OBU2 log file depicting the recorded IMA event
Chapter 8. Test Procedures
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 67
Test Case ID Objective Entrance Criteria Procedure Exit Criteria Data Output
CVE-OBU005-v01 Verify a host Transit Vehicle OBU logs an LCW-based transit vehicle interaction event only when an LCW threat exists
OBU1 is configured for "normal" operation, including broadcasting BSMs
OBU2 (Transit) is configured for "normal" operation, including broadcasting BSMs and logging LCW events
A location is set up as a typical two lane (same direction of travel) roadway segment in the Demonstration Area
Vehicle with OBU1 (V1) drives on the left side of the vehicle with OBU2 (V2) (Transit)
V2 attempts to change lanes to the left
Based on BSMs from OBU1, OBU2 determines a threat exists
Since V2 (Transit) does not contain and HMI, OBU2 only records the LCW event
OBU2 records an LCW event OBU2 log file depicting the recorded LCW event
CVE-OBU006-v01 Verify a host Transit Vehicle OBU logs an RLVW-based transit vehicle interaction event only when an RLVW threat exists
RSU is configured for "normal" operation, including broadcasting SPaT and MAP
OBU1 (Transit) is configured for "normal" operation, including broadcasting BSMs and logging RLVW events
A location is set up as a simple intersection in the Demonstration Area
RSU is broadcasting SPaT and MAP for the intersection
Vehicle with OBU1 (V1) (Transit) approaches the intersection in such a way that the light will be Red when V1 arrives at the intersection
V1 maintains constant speed and heading
Based on SPaT and MAP from the RSU, OBU1 determines a threat exists
Since V1 (Transit) does not contain and HMI, OBU1 only records the RLVW event
OBU1 records an RLVW event OBU1 log file depicting the recorded RLVW event
Chapter 8. Test Procedures
68 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
Test Case ID Objective Entrance Criteria Procedure Exit Criteria Data Output
CVE-OBU007-v01 Verify a host Transit Vehicle OBU logs an RSSZ-based transit vehicle interaction event only when receiving a RSSZ message from an RSU and the host vehicle is in, or approaching, the School Zone traveling above the School Zone speed limit
RSU is configured for "normal" operations, including broadcasting a RSSZ
OBU1 (Transit) is configured for "normal" operation, including broadcasting BSMs and logging RSSZ events
A location is set up as a School Zone (Reduced Speed roadway segment) in the Demonstration Area
RSU is broadcasting RSSZ for the School Zone
Vehicle with OBU1 (V1) (Transit) approaches the School Zone segment above the School Zone Speed Limit
Based on the RSSZ from the RSU, OBU1 determines a threat exists
Since V1 (Transit) does not contain and HMI, OBU1 only records the RSSZ event
OBU1 records an RSSZ event OBU1 log file depicting the recorded RSSZ event
CVE-OBU008-v01 Verify a host Transit Vehicle OBU logs a TSP-based transit vehicle interaction event when it broadcasts an SRM
RSU is configured for "normal" operations, including broadcasting SPaT and MAP and WSAs containing SRM
OBU1 (Transit) is configured for "normal" operation, including broadcasting BSMs and logging TSP events
A location is set up as a simple intersection in the Demonstration Area
RSU is broadcasting SPaT and MAP for the intersection and WSAs advertising SRM on Channel 176
Vehicle with OBU1 (V1) approaches the intersection in such a way that the light will be Red when vehicle arrives at the intersection
Based on the SPaT, MAP, and WSA from the RSU, OBU1 broadcasts an SRM, requesting Signal Priority
OBU1 records the SRM event
OBU1 records an TSP event OBU1 log file containing the TSP event
Chapter 8. Test Procedures
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 69
Test Case ID Objective Entrance Criteria Procedure Exit Criteria Data Output
CVE-OBU009-v01 Verify a Transit vehicle OBU logs all messages (BSM, SPaT, MAP, SRM, SSM, RTCM, RSM) that it receives for a pre-determined period
OBU1 (Transit) is configured for "normal" operation, including logging received messages
A location is set up as a simple intersection in the Demonstration Area
RSU is broadcasting SPaT and MAP for the intersection, WSAs advertising SRM on Channel 176, and RSM, and RTCM
Vehicle with OBU1 (V1) approaches the intersection
OBU1 records SPaT, MAP, RSM, and RTCM
OBU1 records all received messages
OBU1 log file containing received messages
Chapter 8. Test Procedures
70 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
Test Case ID Objective Entrance Criteria Procedure Exit Criteria Data Output
CVE-OBU010-v01 Verify a Transit Vehicle Interaction Event consists of all logged messages for a predetermined period before and after the warning generating the event was issued.
OBU1 is configured for "normal" operation, including broadcasting BSMs
OBU2 (Transit) is configured for "normal" operation, including broadcasting BSMs and logging events
Values (in seconds) are configured in the Transit OBU for retaining logged messages before an event (x) and after an event (y) is detected
A location is set up as a typical roadway segment in the Demonstration Area
Vehicle with OBU1 (V1) leads vehicle with OBU2 (V2) (Transit) on the roadway
V1 performs a hard-braking maneuver such that an EEBL event flag is set in its BSMs
Based on BSMs from OBU1, OBU2 determines a threat exists
Since V2 (Transit) does not contain and HMI, OBU2 only records the EEBL event along with all other CV messages received for x seconds before the EEBL Event and y seconds after the EEBL Event
OBU2 records an EEBL event along with all other CV messages received for x seconds before the EEBL event and y seconds after the EEBL event
OBU2 log file depicting the recorded EEBL event
Chapter 8. Test Procedures
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 71
Test Case ID Objective Entrance Criteria Procedure Exit Criteria Data Output
CVE-OBU011-v01 Verify a Transit Vehicle OBU uploads Transit Vehicle Interaction Event Data to the TrCVMS when arriving to the COTA garage, and deletes local transit vehicle interaction events after uploading
RSU is configured for "normal" operations
RSU is connected to the TrCVMS
OBU1 (Transit) is configured for "normal" operation, including broadcasting BSMs and offloading log files
TrCVMS is configured for normal operations, including receiving log files from Transit vehicles
RSU is broadcasting WSA advertising Log Offload service
OBU1 approaches RSU and begins offloading its log files to the TrCVMS
OBU1 deletes its log files, once off loaded
Verify log files are archived in the TrCVMS
Verify OBU1 deleted its log files
TrCVMS contains OBU1 log files
OBU1 deleted log files
Screen shot, including date and time stamp, of TrCVMS file directory depicting OBU1 log files
Screen shot, including date and time stamp, of empty OBU1 log file directory
Chapter 8. Test Procedures
72 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
Test Case ID Objective Entrance Criteria Procedure Exit Criteria Data Output
CVE-OBU012-v01 Verify the EV OBU broadcasts SRMs only when lights and sirens are active, and it is approaching an equipped intersection
RSU is configured for "normal" operations, including broadcasting SPaT and MAP and WSAs advertising SRM
OBU1 (EV) is configured for "normal" operation, including broadcasting BSMs and SRMs when the lights and sirens are on
CVTT is configured to capture DSRC packets
A location is set up as a simple intersection in the Demonstration Area
RSU is broadcasting SPaT and MAP for an intersection in the Demonstration Area and WSAs advertising SRM on Channel 176
CVTT is configured to capture packets on the Control Channel (178) and Service Channel 176
OBU1 is installed in an EV connected to the lights and sirens
EV approaches the Demonstration Area intersection with its lights and sirens off (not requesting preemption)
EV approaches the Demonstration Area intersection with its lights and sirens on in such a way that OBU1 broadcasts SRMs (to request preemption)
OBU1 only broadcasts SRMs when approaching the intersection advertising SRM when the EV lights and sirens are on
CVTT pcap file containing WSAs and SRMs
CVVT pcap file depicting the EV only broadcast SRMs when SRM is advertised and the lights and sirens are on
Chapter 8. Test Procedures
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 73
Test Case ID Objective Entrance Criteria Procedure Exit Criteria Data Output
CVE-OBU013-v01 Verify the EV OBU receives an SSM
RSU is configured for "normal" operations, including broadcasting SPaT, MAP, SSMs, and WSAs advertising SRM
RSU is connected to a Message Handler sending J2735 SSMs to the RSU
Message Handler is connected to a Signal Controller sending signal status data to the Message Handler
OBU1 (EV) is configured for "normal" operation, including broadcasting BSMs, broadcasting SRMs, and logging SSMs
CVTT is configured to capture DSRC packets
RSU is broadcasting SPaT and MAP for an intersection in the Demonstration Area and WSAs advertising SRM on Channel 176
CVTT is configured to capture packets on the Control Channel (178) and Service channel 176
OBU1 is installed in an EV connected to the lights and sirens
EV approaches the Demonstration Area intersection with its lights and sirens on in such a way that OBU1 broadcasts SRMs (to request preemption)
RSU forwards SRMs to Message Handler
Message Handlers sends a Preemption Call to the Signal Controller
Signal Controller sends signal status data to the Message Handler
Message Handler sends SSMs to the RSU
RSU broadcast SSMs
OBU1 receives and logs SSMs
OBU1 logs received SSMs OBU1 log file containing SSMs
CVTT pcap file containing WSAs, SRMs, and SSMs
Chapter 8. Test Procedures
74 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
Test Case ID Objective Entrance Criteria Procedure Exit Criteria Data Output
CVE-OBU014-v01 Verify the Transit Vehicle OBU broadcasts SRMs only when approaching an equipped intersection
RSU is configured for "normal" operations, including broadcasting SPaT and MAP and WSAs advertising SRM
OBU1 (Transit) is configured for "normal" operation, including broadcasting BSMs and SRMs
CVTT is configured to capture DSRC packets
A location is set up as a simple intersection in the Demonstration Area
RSU is broadcasting SPaT and MAP for an intersection in the Demonstration Area and WSAs advertising SRM on Channel 176
CVTT is configured to capture packets on the Control Channel (178) and Service channel 176
OBU1 drives around the Demonstration Area, avoiding the intersection (not requesting priority)
OBU1 approaches the Demonstration Area intersection in such a way that OBU1 broadcasts SRMs (to request priority)
OBU1 only broadcasts SRMs when approaching the intersection advertising SRM
CVTT pcap file containing WSAs and SRMs
CVVT pcap file depicting OBU1 only broadcast SRMs when SRM is advertised
Chapter 8. Test Procedures
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 75
Test Case ID Objective Entrance Criteria Procedure Exit Criteria Data Output
CVE-OBU015-v01 Verify the Transit Vehicle OBU receives an SSM
RSU is configured for "normal" operations, including broadcasting SPaT, MAP, SSMs, and WSAs advertising SRM
RSU is connected to a Message Handler sending J2735 SSMs to the RSU
Message Handler is connected to a Signal Controller sending signal status data to the Message Handler
OBU1 (Transit) is configured for "normal" operation, including broadcasting BSMs, broadcasting SRMs, and logging SSMs
CVTT is configured to capture DSRC packets
A location is set up as a simple intersection in the Demonstration Area
RSU is broadcasting SPaT and MAP for an intersection in the Demonstration Area and WSAs advertising SRM on Channel 176
CVTT is configured to capture packets on the Control Channel (178) and Service channel 176
OBU1 approaches the Demonstration Area intersection in such a way that OBU1 broadcasts SRMs (to request priority)
RSU forwards SRMs to Message Handler
Message Handlers sends a Priority Call to the Signal Controller
Signal Controller sends signal status data to the Message Handler
Message Handler sends SSMs to the RSU
RSU broadcast SSMs
OBU1 receives and logs SSMs
OBU1 logs received SSMs OBU1 log file containing SSMs
CVTT pcap file containing WSAs, SRMs, and SSMs
Chapter 8. Test Procedures
76 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
Test Case ID Objective Entrance Criteria Procedure Exit Criteria Data Output
CVE-OBU016-v01 Verify the HDV OBU broadcasts SRMs only when approaching an equipped intersection
RSU is configured for "normal" operations, including broadcasting SPaT and MAP and WSAs advertising SRM
OBU1 (HDV) is configured for "normal" operation, including broadcasting BSMs and SRMs
CVTT is configured to capture DSRC packets
A location is set up as a simple intersection in the Demonstration Area
RSU is broadcasting SPaT and MAP for an intersection in the Demonstration Area and WSAs advertising SRM on Channel 176
CVTT is configured to capture packets on the Control Channel (178) and Service channel 176
OBU1 drives around the Demonstration Area, avoiding the intersection (not requesting priority)
OBU1 approaches the Demonstration Area intersection in such a way that OBU1 broadcasts SRMs (to request priority)
OBU1 only broadcasts SRMs when approaching the intersection advertising SRM
CVTT pcap file containing WSAs and SRMs
CVVT pcap file depicting OBU1 only broadcast SRMs when SRM is advertised
Chapter 8. Test Procedures
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 77
Test Case ID Objective Entrance Criteria Procedure Exit Criteria Data Output
CVE-OBU017-v01 Verify the HDV OBU receives an SSM
RSU is configured for "normal" operations, including broadcasting SPaT, MAP, SSMs, and WSAs advertising SRM
RSU is connected to a Message Handler sending J2735 SSMs to the RSU
Message Handler is connected to a Signal Controller sending signal status data to the Message Handler
OBU1 (HDV) is configured for "normal" operation, including broadcasting BSMs, broadcasting SRMs, and logging SSMs
CVTT is configured to capture DSRC packets
A location is set up as a simple intersection in the Demonstration Area
RSU is broadcasting SPaT and MAP for an intersection in the Demonstration Area and WSAs advertising SRM on Channel 176
CVTT is configured to capture packets on the Control Channel (178) and Service channel 176
OBU1 approaches the Demonstration Area intersection in such a way that OBU1 broadcasts SRMs (to request priority)
RSU forwards SRMs to Message Handler
Message Handlers sends a Priority Call to the Signal Controller
Signal Controller sends signal status data to the Message Handler
Message Handler sends SSMs to the RSU
RSU broadcast SSMs
OBU1 receives and logs SSMs
OBU1 logs received SSMs OBU1 log file containing SSMs
CVTT pcap file containing WSAs, SRMs, and SSMs
Source: City of Columbus
Chapter 8. Test Procedures
78 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
8.2. ONBOARD UNIT APPLCIATION
Table 14: Onboard Unit Application Test Procedures
Test Case ID Objective Entrance Criteria Procedure Exit Criteria Data Output
CVE-App001-v01 Verify a host LDV OBU issues an EEBL warning when it receives a BSM from a remote OBU (under conditions in which an EEBL warning should be issued)
OBU1 is configured for "normal" operation, including broadcasting BSMs
OBU2 is configured for "normal" operation, including broadcasting BSMs and providing EEBL alerts
A location is set up as a normal roadway segment in the Demonstration Area
Vehicle with OBU1 (V1) leads vehicle with OBU2 (V2) on the road segment
V1 performs a hard braking maneuver such that an EEBL event flag is set in OBU1s BSM.
Based on BSMs from OBU1, OBU2 determines a threat exists, and issues an EEBL alert to the driver
Data Recorder logs an EEBL alert in the Test Log
OBU2 issues an EEBL alert OBU2 log file depicting an EEBL alert was issued
Test Log indicating EEBL alert was issued
CVE-App002-v01 Verify a host LDV OBU issues an FCW warning when it receives a BSM from a remote OBU (under conditions in which an FCW warning should be issued)
OBU1 is configured for "normal" operation, including broadcasting BSMs
OBU2 is configured for "normal" operation, including broadcasting BSMs and providing FCW alerts
A location is set up as a normal roadway segment in the Demonstration Area
Vehicle with OBU1 (V1) is stopped on the roadway segment
Vehicle with OBU2 (V2) approaches V1 from behind
Base on BSMs from OBU1, OBU2 determines a threat exists and issues an FCW alert
Data Recorder logs an FCW alert in the Test Log
OBU2 issues an FCW alert OBU2 log file depicting an FCW alert was issued
Test Log indicating FCW alert was issued
Chapter 8. Test Procedures
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 79
Test Case ID Objective Entrance Criteria Procedure Exit Criteria Data Output
CVE-App003-v01 Verify a host LDV OBU issues an LCW warning when it receives a BSM from a remote OBU (under conditions in which an LCW warning should be issued)
OBU1 is configured for "normal" operation, including broadcasting BSMs
OBU2 is configured for "normal" operation, including broadcasting BSMs and providing LCW alerts
A location is set up as a typical two lane (same direction of travel) roadway segment in the Demonstration Area
Vehicle with OBU1 (V1) drives next to the vehicle with OBU2 (V2), on the left side
V2 attempts to change lanes to the left
Based on BSMs from OBU1, OBU2 determines a threat exists and issues an LCW alert
Data Recorder logs an LCW alert in the Test Log
OBU2 issues an LCW alert OBU2 log file depicting an LCW alert was issued
Test Log indicating LCW alert was issued
CVE-App004-v01 Verify a host LDV OBU issues an IMA warning when it receives a BSM from a remote OBU (under conditions in which an IMA warning should be issued)
OBU1 is configured for "normal" operation, including broadcasting BSMs
OBU2 is configured for "normal" operation, including broadcasting BSMs and providing IMA alerts
A location is set up as a normal two lane (opposite direction of travel) roadway segment in the Demonstration Area
Vehicle with OBU1 (V1) approaches vehicle with OBU2 (V2) in the lane to the left of V2
V2 begins to turn left in front of V1
Based on BSMs from OBU1, OBU2 issues an IMA alert
Data Recorder logs an IMA alert in the Test Log
OBU2 issues an IMA alert OBU2 log file depicting an IMA alert was issued
Test Log indicating IMA alert was issued
Chapter 8. Test Procedures
80 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
Test Case ID Objective Entrance Criteria Procedure Exit Criteria Data Output
CVE-App005-v01 Verify a host LDV OBU issues an RLVW warning when it receives SPaT and MAP from an RSU (under conditions in which an RLVW warning should be issued)
RSU is configured for "normal" operation, including broadcasting SPaT and MAP
OBU1 is configured for "normal" operation, including broadcasting BSMs and providing RLVW alerts
A location is set up as a simple intersection in the Demonstration Area
RSU is broadcasting SPaT and MAP for the intersection
Vehicle with OBU1 (V1) approaches the intersection in such a way that the light will be Red when V1 arrives at the intersection
V1 maintains constant speed and heading
Based on SPaT and MAP from the RSU, OBU1 determines a threat exists and issues a RLVW alert
Data Recorder logs an RLVW alert in the Test Log
OBU1 issues an RLVW alert OBU1 log file depicting an RLVW alert was issued
Test Log indicating RLVW alert was issued
CVE-App006-v01 Verify a host LDV OBU issues an RSSZ warning when it receives an RSM from an RSU (under conditions in which an RSSZ warning should be issued)
RSU is configured for "normal" operations, including broadcasting a RSSZ
OBU1 is configured for "normal" operation, including broadcasting BSMs and providing RSSZ alerts
A location is set up as a School Zone (Reduced Speed roadway segment) in the Demonstration Area
RSU is broadcasting RSSZ for the School Zone
Vehicle with OBU1 (V1) approaches the School Zone segment above the School Zone Speed Limit
Based on the RSSZ from the RSU, OBU1 determines a threat exists and issues and RSSZ alert
Data Recorder logs an RSSZ alert in the Test Log
OBU1 issues an RSSZ alert OBU1 log file depicting an RSSZ alert was issued
Test Log indicating RSSZ alert was issued
Chapter 8. Test Procedures
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 81
CVE-App007-v01 Verify the EV OBU provides notification to driver when preemption is granted
RSU is configured for "normal" operations, including broadcasting SPaT, MAP, SSMs, and WSAs advertising SRM
RSU is connected to a Message Handler sending J2735 SSMs to the RSU
Message Handler is connected to a Signal Controller sending signal status data to the Message Handler
OBU1 (EV) is configured for "normal" operation, including broadcasting BSMs, broadcasting SRMs, and logging SSMs
CVTT is configured to capture DSRC packets
A location is set up as a simple intersection in the Demonstration Area
RSU is broadcasting SPaT and MAP for the intersection and WSAs advertising SRM on Channel 176
OBU1 is installed in an EV connected to the lights and sirens
EV approaches the intersection with its lights and sirens on in such a way that the light will be Red when EV arrives at the intersection
Based on the SPaT, MAP, and WSA from the RSU, OBU1 broadcasts an SRM, requesting Signal Preemption
RSU forwards SRMs to Message Handler
Message Handlers sends a Preemption Call to the Signal Controller
Signal Controller sends signal status data to the Message Handler
Message Handler sends SSMs to the RSU
RSU broadcast SSMs
OBU1 receives SSMs and issues a "preemption granted" notification
Data Recorder logs the "preemption granted" notification in the Test Log
OBU1 issues "preemption granted" notification
OBU2 log file depicting a "preemption granted" notification was issued
Test Log indicating "preemption granted" notification was issued
Chapter 8. Test Procedures
82 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
Test Case ID Objective Entrance Criteria Procedure Exit Criteria Data Output
CVE-App008-v01 Verify a high-priority warning takes precedence over a low priority warning when both warning types are present.
OBU1 is configured for "normal" operation, including broadcasting BSMs and displaying RSSZ alerts
OBU2 is configured for "normal" operation, including broadcasting BSMs and displaying EEBL and RSSZ alerts
RSU is configured for "normal" operations, including broadcasting a RSSZ
A location is set up as a School Zone (Reduced Speed roadway segment) in the Demonstration Area
RSU is broadcasting RSSZ for the intersection
Vehicle with OBU1 (V1) leads vehicle with OBU2 (V2) in the School Zone driving above the School Zone Speed Limit
Both V1 and V2 drivers receive RSSZ alerts
While in the School Zone, V1 performs a hard-braking maneuver such that an EEBL event is contained its BSM.
Verify the HMI in V2 only shows an EEBL warning
V2 only shows EEBL alert with and EEBL and RSSZ alerts present
OBU2 log file depicting the RSSZ (lower priority) alert was suppressed for an EEBL (higher priority) alert
Source: City of Columbus
Chapter 8. Test Procedures
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 83
8.3. ROADSIDE UNIT
Table 15: Roadside Unit Test Procedures
Test Case ID Objective Entrance Criteria Procedure Exit Criteria Data Output
CVE-RSU001-v01 Verify RSU forwards BSMs, received from passing vehicles, to TCVMS.
RSU is configured for "normal" operation, including forwarding BSMs to the TCVMS
RSU is connected to a network capable of accessing the TCVMS
OBU is configured for "normal" operation, including broadcasting BSMs
TCVMS is configured for "normal" operations, including accepting BSMs from RSUs
TCVMS is connected to a network capable of accessing the RSU
OBU, broadcasting BSMs, is brought into communication range of the RSU Under Test
Monitor the TCVM to verify it is receiving BSMs
TCVMS successfully receives BSMs from the RSU/OBU
OBU log file containing transmitted BSMs
TCVMS interface containing BSMs
CVE-RSU002-v01 Verify RSU forwards SRMs, received from approaching vehicles, to the Message Handler
RSU is configured for "normal" operation, including forwarding SRMs to the Message Handler
RSU is connected to a Message Handler
OBU is configured to broadcast SRMs continually (or on-demand)
OBU, broadcasting SRMs, is brought into communication range of the RSU Under Test
Monitor the Message Handler to verify it is receiving SRMs
Message Handler successfully receives SRMs
OBU log file containing SRMs
Message Handler log file containing SRMs
Chapter 8. Test Procedures
84 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
Test Case ID Objective Entrance Criteria Procedure Exit Criteria Data Output
CVE-RSU003-v01 Verify RSUs broadcast SPaT messages received from the Message Handler over DSRC on Channel 172 at 10Hz
RSU is configured for "normal" operation, including broadcasting SPaT using the Immediate Forward functionality
RSU is connected to a Message Handler sending J2735 SPaT Messages
Begin broadcasting SPaT from the Message Handler
Using the CVTT message capture functionality, monitor the RSU to verify it is broadcasting SPaT based on the messages provided by the Message Handler
CVTT successfully captures SPaT Messages from the RSU
CVTT pcap file containing SPaT
CVE-RSU004-v01 Verify SPaT broadcast by RSUs match the Signal Head Status
RSU is configured for "normal" operation, including broadcasting SPaT and MAP
RSU is connected to a Message Handler sending J2735 SPaT Messages
RSU has a MAP file stored for broadcast
Begin broadcasting SPaT from the Message Handler
Begin RSU broadcasting SPaT and MAP
Using the CVTT SPaT/MAP visualization functionality, monitor the CVTT to verify SPaT status matches the Signal Head status for all lanes
SPaT status matches the Signal head status
CVTT log file containing SPaT and MAP messages
CVE-RSU005-v01 Verify RSUs store MAP files received from the TCVMS
RSU is configured for "normal" operation, including accepting MAP files from the TCVMS
RSU is connected to a network capable of accessing the TCVMS
TCVMS is configured for "normal" operations, including sending MAP files to RSUs
TCVMS is connected to a network capable of accessing the RSU
A MAP file is loading on the TCVMS for the RSU under test
From TCVMS, send MAP file to RSU
Inspect RSU to verify the MAP file is loading in the appropriate location (directory) and enabled for broadcast
MAP file is successfully loading on the RSU from the TCVMS
MAP file from TCVMS
Screen shot of RSU directory containing the MAP file
Chapter 8. Test Procedures
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 85
Test Case ID Objective Entrance Criteria Procedure Exit Criteria Data Output
CVE-RSU006-v01 Verify RSUs broadcast MAP messages over DSRC on Channel 172 at 1Hz
RSU is configured for "normal" operation, including broadcasting MAP files using the Store and Repeat functionality
RSU contains a MAP file for its installation location
Begin broadcasting MAP from the RSU
Using the CVTT message capture functionality, monitor the RSU to verify it is broadcasting MAP based on the message stored on the device
CVTT successfully captures MAP messages from the RSU
CVTT pcap file containing MAP messages
CVE-RSU007-v01 Verify MAP file matches intersection geometry
RSU is configured for "normal" operation, including broadcasting MAP files using the Store and Repeat functionality
RSU contains a MAP file for its installation location
Begin broadcasting MAP from the RSU
Using the CVTT message capture functionality, monitor the RSU to verify it is broadcasting MAP based on the message stored on the device
Using the CVTT MAP display functionality verify the MAP files matches/aligns with the RSU installation location
The MAP message displayed by the CVTT matches the lanes and approaches for RSUs installation location
Screen shot of the MAP displayed by the CVTT
CVE-RSU008-v01 Verify RSUs broadcast RTCM messages received from the Message Handler over DSRC on the specified Service Channel at the specified rate
RSU is configured for "normal" operation, including broadcasting RTCMs using the Immediate Forward functionality
RSU is connected to a Message Handler sending J2735 RTCM messages
Begin broadcasting RTCMs from the Message Handler
Using the CVTT message capture functionality, monitor the RSU to verify it is broadcasting RTCMs based on the messages provided by the Message Handler
CVTT successfully captures RTCM Messages from the RSU
CVTT pcap file containing RTCM
Chapter 8. Test Procedures
86 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
Test Case ID Objective Entrance Criteria Procedure Exit Criteria Data Output
CVE-RSU009-v01 Verify RSUs broadcasts SSM received from the Message Handler over DSRC on the specified Service Channel at the specified rate
RSU is configured for "normal" operation, including broadcasting SSMs using the Immediate Forward functionality
RSU is connected to a Message Handler sending J2735 SSM messages
Begin broadcasting SSMs from the Message Handler
Using the CVTT message capture functionality, monitor the RSU to verify it is broadcasting SSM based on the messages provided by the Message Handler
CVTT successfully captures SSMs from the RSU
CVTT pcap file containing SSMs
CVE-RSU010-v01 Verify RSUs broadcasts RSMs at 1 Hz when the School Zone signal is flashing.
RSU is configured for "normal" operation, including broadcasting RSMs using the Immediate Forward functionality
RSU is connected to the Message Handler
Message Handler is configured for "normal" operation, including detecting when the School Zone signal is flashing and sending RSMs to the RSU using the Immediate Forward function
CVTT is configured to capture DSRC packets
Connect a toggle switch (S1) to the School Zone flasher Input/output (IO) port of the Message Handler
Set S1 to "inactive" (School Zone flasher is NOT active)
Power on the RSU and Message Handler
Using the CVTT, monitor the RSU to verify it is NOT broadcasting RSMs (for RSSZ)
Set S1 to "active" (School Zone flasher is active)
Once School Zone flasher is detected as active, the Message Handler begins sending RSM (for RSSZ) to the RSU for broadcast
RSU begins broadcasting RSM (for RSSZ)
Using the CVTT, monitor the RSU to verify it is broadcasting RSMs
RSU broadcasts RSM (for RSSZ) only when the School Zone flasher is active
CVTT pcap NOT containing RSMs (for RSSZ) when the School Zone flasher is NOT active
CVTT pcap containing RSMs (for RSSZ) when the School Zone flasher is active
Chapter 8. Test Procedures
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 87
Test Case ID Objective Entrance Criteria Procedure Exit Criteria Data Output
CVE-RSU011-v01 Verify RSUs report Channel Busy Ratio to TCVMS, when above a predetermined threshold
RSU (1) is configured for "normal" operation, including broadcasting WSA, MAP, and RSM
RSU (2) is configured for "normal" operation, including detecting and reporting Channel Busy Ratio
TCVMS is configured for "normal" operation, including receiving and logging Channel Busy Ratio
RSU (2) is configured for a Channel Busy Raito threshold of 5% (which should trigger a Channel Busy Report with minimal DSRC activity)
Using the CVTT message capture functionality capture WSA, MAP, and RSMs from RSU (1)
Begin broadcasting WSA, MAP, RSM from RSU (1)
Monitor RSU (2) to verify it detects a Channel Busy Ratio above 5% and sends a Channel Busy Report to the TCVMS
RSU (2) sends Channel Busy Report to TCVMS
RSU log file containing the Channel Busy Report indicating Channel Busy Raito is above 5%, sent to the TCVMS
TCVM log file containing Channel Busy Report indicating Channel Busy Raito is above 5%
CVTT pcap file containing WSA, MAP, and RSM from RSU (1)
CVE-RSU012-v01 Verify RSU broadcast WSA with correct parameters (including WRA)
RSU is configured for "normal" operation, including broadcasting WSAs containing WRAs
Begin broadcasting WSAs from the RSU
Using the CVTT message capture functionality, monitor the RSU to verify it is broadcasting WSAs containing the correct parameters, including the WRA
CVTT successfully captures WSA containing the correct parameters, including the WRA
CVTT pcap file containing WSAs
CVE-RSU013-v01 Verify IPv6 connectivity to SCMS
RSU is configured for "normal" operation, including requesting new certificates from the SCMS
RSU is connected to a network capable of accessing the SCMS
Remove certificates from the RSU
From the RSU, initiate a Certificate Request to the SCMS
Monitor the certificate directory on the RSU to verify the RSU has received and stored new certificates
RSU successfully requests, receives, and stores certificates from the SCMS
Screen shot of the RSU directory containing the new certificates
Source: City of Columbus
Chapter 8. Test Procedures
88 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
8.4. MESSAGE HANDLER
Table 16: Message Handler Test Procedures
Test Case ID Objective Entrance Criteria Procedure Exit Criteria Data Output
CVE-MH001-v01 Verify the Message Handler generates and forwards SAE J2735-2016 SPaT messages, based on SPaT data received from the local signal controller, to the local RSU
Message Handler is configured for "normal" operation, including generating J2735 SPaT messages and sending them to an RSU
Message Handler is connected to a network capable of accessing the Signal Controller
Signal Controller is configured for "normal" operation, including sending SPaT data to the Message Handler
Signal Controller is connected to a network capable of accessing the Message Handler
Connect a laptop (L1) running Wireshark between the Signal Controller and the Message Handler such that data can flow between the Signal Controller and the Message Handler and Wireshark can monitor the data packets.
Log received packets on L1
Power on the Message Handler
Power on the Signal Controller
Connect a second laptop (L2) running Wireshark to the RSU port on the Message Handler (L2 acts as the RSU)
Configure Wireshark on L2 to capture packets from the Message Handler
Log received packets on L2
Monitor Wireshark on L1 to be sure data is flowing between the signal controller and the Message Handler
Inspect packets coming from the Message Handler on L2 to verify they conform to J2735 SPaT messages
Wireshark captures J2735 SPaT messages from the Message Handler
pcap file from L2 containing SPaT messages sent by the Message Handler
pcap file from L1 containing data sent to the Message Handler by the Signal Controller
Chapter 8. Test Procedures
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 89
Test Case ID Objective Entrance Criteria Procedure Exit Criteria Data Output
CVE-MH002-v01 Verify the Message Handler properly determines if OBU requesting priority/preemption is authorized to receive priority/preemption
Message Handler is configured for "normal" operation, including distinguishing between authorized priority/preemption requests and unauthorized priority/preemption requests based on the 1609.2 certificate
Message Handler is connected to a network capable of accessing the RSU
RSU is configured for "normal" operation, including forwarding J2735 SRMs to the Message Handler
RSU is connected to a network capable of accessing the Message Handler
OBU1 is configured to broadcast SRMs signed with a signal priority certificate, requesting signal priority (authorized)
OBU2 is configured to broadcast SRMs signed with a signal priority certificate, requesting signal preemption (unauthorized)
Power on RSU and Message Handler
Power on OBU1 and OBU2
OBU1 broadcasts SRMs signed with a signal priority certificate, requesting signal priority (authorized)
OBU2 broadcasts SRMs signed with a signal priority certificate, requesting signal preemption (unauthorized)
RSU forwards SRMs from both OBU1 and OBU2 to the Message Handler
Message Handler logs authorized signal priority requests and unauthorized signal preemption requests
Message Handler authorizes OBU1 to request priority
Message Handler denies OBU2 from requesting priority
Message Handler log file identifying authorized and unauthorized priority requests
Chapter 8. Test Procedures
90 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
Test Case ID Objective Entrance Criteria Procedure Exit Criteria Data Output
CVE-MH003-v01 Verify the Message Handler properly arbitrates priority/preemption requests when receiving multiple valid requests.
Message Handler is configured for "normal" operation, including distinguishing between authorized priority and preemption requests based on the 1609.2 certificate
Message Handler is connected to a network capable of accessing the RSU
RSU is configured for "normal" operation, including forwarding J2735 SRMs to the Message Handler
RSU is connected to a network capable of accessing the Message Handler
OBU1 is configured to broadcast authorized J2735 SRMs requesting preemption
OBU2 is configured to broadcast authorized J2735 SRMs requesting priority
Power on RSU and Message Handler
Power on OBU1 and OBU2
OBU1 broadcasts SRMs signed with a signal preemption certificate
OBU2 broadcasts SRMs signed with a signal priority certificate
RSU forwards SRMs from both OBU1 and OBU2 to the Message Handler
Message Handler logs status data indicating Signal Preemption request is "active" and Signal Priority requests are "in queue"
Message Handler authorizes OBU1 for preemption request
Message Handler log file identifying authorized preemption requests as active from OBU1 and authorized priority requests as in-queue form OBU2
Chapter 8. Test Procedures
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 91
Test Case ID Objective Entrance Criteria Procedure Exit Criteria Data Output
CVE-MH004-v01 Verify the Message Handler places a priority call to the local Signal Controller based on an authorized SRM.
Message Handler is configured for "normal" operation, including receiving SRMs from an RSU and placing a priority call to a Signal Controller
Message Handler is connected to a network capable of accessing the RSU
RSU is configured for "normal" operation, including forwarding J2735 SRMs to the Message Handler
RSU is connected to a network capable of accessing the Message Handler
OBU1 is configured to broadcast authorized J2735 SRMs requesting priority
Power on RSU and Message Handler
Connect a laptop (L1) running Wireshark to the Signal Controller port of the Message Handler (L1 acts as the Signal Controller)
Configure Wireshark on L1 to capture packets from the Message Handler
Log received packets on L1
Power on OBU1
OBU1 broadcasts SRMs signed with a signal priority certificate
In Wireshark on L1, inspect packets coming from the Message Handler to verify received packets contain Priority Calls from OBU1
Message Handler logs signal priority requests from OBU1 and Signal Priority Calls sent to the Signal Controller (L1)
Message Handler sends Priority Calls from OBU1 to the Signal Controller
Message Handler log file containing SRMs received from the RSU and Priority Calls sent to the Signal Controller
Wireshark pcap files containing Signal Priority Calls from OBU1
Chapter 8. Test Procedures
92 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
Test Case ID Objective Entrance Criteria Procedure Exit Criteria Data Output
CVE-MH005-v01 Verify the Message Handler places a preemption call to the local signal controller based on an authorized SRM.
Message Handler is configured for "normal" operation, including receiving SRMs from and RSU and placing a preemption call to a Signal Controller
Message Handler is connected to a network capable of accessing the RSU
RSU is configured for "normal" operation, including forwarding J2735 SRMs to the Message Handler
RSU is connected to a network capable of accessing the Message Handler
OBU1 is configured to broadcast authorized J2735 SRMs requesting preemption
Power on RSU and Message Handler
Connect a laptop (L1) running Wireshark to the Signal Controller port of the Message Handler (L1 acts as the Signal Controller)
Configure Wireshark on L1 to capture packets from the Message Handler
Log received packets on L1
Power on OBU1
OBU1 broadcasts SRMs signed with a signal preemption certificate
In Wireshark on L1, inspect packets coming from the Message Handler to verify received packets contain Preemption Calls from OBU1
Message Handler logs signal preemption requests from OBU1 and Signal Preemption Calls sent to the Signal Controller (L1)
Message Handler sends Preemption Calls from OBU1 to the Signal Controller (L1)
Message Handler log file containing SRMs received from the RSU and Preemption Calls sent to the Signal Controller (L1)
Wireshark pcap files containing Signal Preemption Calls from OBU1
Chapter 8. Test Procedures
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 93
Test Case ID Objective Entrance Criteria Procedure Exit Criteria Data Output
CVE-MH006-v01 Verify the Message Handler generates and forwards SAE J2735-2016 SSMs, based on status data received from the local Signal Controller, to the local RSU
Message Handler is configured for "normal" operation, including generating J2735 SSM messages and sending them to an RSU
Message Handler is connected to a network capable of accessing the Signal Controller
Signal Controller is configured for "normal" operation, including sending priority/preemption request status data to the Message Handler
Signal Controller is connected to a network capable of accessing the Message Handler
Connect a laptop (L1) running Wireshark between the Signal Controller and the Message Handler such that data can flow between the Signal Controller and the Message Handler and Wireshark can monitor the data packets.
Log received packets on L1
Power on the Message Handler
Power on the Signal Controller
Connect a second laptop (L2) running Wireshark to the RSU port on the Message Handler (L2 acts as the RSU)
Configure Wireshark on L2 to capture packets from the Message Handler
Log received packets on L2
Monitor Wireshark on L1 to be sure data is flowing between the Signal Controller and the Message Handler
Inspect packets coming from the Message Handler on L2 to verify they conform to J2735 SSM messages
Wireshark captures J2735 SPaT messages from the Message Handler
pcap file from L2 containing SPaT messages sent by the Message Handler
pcap file from L1 containing data sent to the Message Handler by the Signal Controller
Chapter 8. Test Procedures
94 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
Test Case ID Objective Entrance Criteria Procedure Exit Criteria Data Output
CVE-MH007-v01 Verify the Message Handler reports a cabinet tamper events to TCVMS
Message Handler is configured for "normal" operation, including detecting Cabinet Tamper Events and sending events to the TCVMS
Message Handler is connected to a network capable of accessing the TCVMS
TCVMS is configured for "normal" operation, including receiving Cabinet Tamper Events
TCVMS is connected to a network capable of accessing the Message Handler
Connect a toggle switch (S1) to the Cabinet tamper detection Input/output (IO) port of the Message Handler
Set S1 to "inactive" (no event detected)
Connect a laptop (L1) running Wireshark to the TCVMS port on the Message Handler (L1 acts as the TCVMS)
Log received packets on L1
Power on the Message Handler
In Wireshark on L1, inspect packets coming from the Message Handler in Wireshark to verify the Message Handler is NOT sending event messages
Set S1 to "active" (event detected)
In Wireshark on L1, inspect packets coming from the Message Handler to verify the Message Handler is sending event messages
Message Handler logs events based on S1 set to "active"
Wireshark captures event messages from the Message Handler when an event is detected
Message Handler log file containing event messages sent to the TCVMS
Wireshark pcap files containing event messages sent by the Message Handler to the TCVMS
Chapter 8. Test Procedures
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 95
Test Case ID Objective Entrance Criteria Procedure Exit Criteria Data Output
CVE-MH008-v01 Verify the Message Handler generates and forwards SAE J2735-2016 RTCM messages, based on data received from the POSITION CORRECTION SYSTEM
Message Handler is configured for "normal" operation, including generating J2735 RTCM messages and sending them to an RSU
Message Handler is connected to a network capable of accessing a RTCM source
Connect a laptop (L1) running Wireshark between the RTCM source and the Message Handler such that data can flow between the RTCM source and the Message Handler and Wireshark can monitor the data packets.
Log received packets on L1
Power on the Message Handler
Power on the Signal Controller
Connect a second laptop (L2) running Wireshark to the RSU port on the Message Handler (L2 acts as the RSU)
Configure Wireshark on L2 to capture packets from the Message Handler
Log received packets on L2
Monitor Wireshark on L1 to be sure data is flowing between the RTCM source and the Message Handler
Inspect packets coming from the Message Handler on L2 to verify they conform to J2735 RTCM messages
Wireshark captures J2735 RTCM messages from the Message Handler
pcap file from L2 containing RTCM messages sent by the Message Handler
pcap file from L1 containing data sent to the Message Handler by the RTCM source
Source: City of Columbus
Chapter 8. Test Procedures
96 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
8.5. TRAFFIC SIGNAL CONTROLLER
Table 17: Traffic Signal Controller Test Procedures
Test Case ID Objective Entrance Criteria Procedure Exit Criteria Data Output
CVE-TSC001-v01 Verify Traffic Signal Controller sends SPaT data to the Message Handler
Signal Controller is configured for "normal" operation, including sending SPaT data to the Message Handler
Connect a laptop (L1) running Wireshark between the Signal Controller Message Handler port (L1 acts as Message Handler)
Log received packets on L1
Power on the Signal Controller
Inspect packets coming from the Signal Controller on L1 to verify the Signal Controller is send SPaT data
Wireshark captures J2735 SPaT data from the Signal Controller
pcap file from L1 containing SPaT data sent from the Signal Controller to the Message Handler
Chapter 8. Test Procedures
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 97
Test Case ID Objective Entrance Criteria Procedure Exit Criteria Data Output
CVE-TSC002-v01 Verify Traffic Signal Controller processes a Signal Priority Call from the Message Handler
Message Handler is configured for "normal" operation, including receiving SRMs from an RSU and placing a priority call to a Signal Controller
Message Handler is connected to the RSU
RSU is configured for "normal" operation, including forwarding J2735 SRMs to the Message Handler
Signal Controller is configured for "normal" operation including processing Priority Calls from the Message Handler
Signal Controller is connected to the Message Handler
OBU1 is configured to broadcast authorized J2735 SRMs requesting priority
Power on RSU, Message Handler, OBU1, and Signal Controller
OBU1 broadcasts SRMs signed with a signal priority certificate
RSU forwards SRM to the Message Handler
Message Handler sends a Signal Priority Call to the Signal Controller
Signal Controller changes timing based on the priority call
Data Recorder log the time change in the test log
Signal Controller change timing based on the priority call
Test Log indicating Signal Controller changes timing appropriately
Chapter 8. Test Procedures
98 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
Test Case ID Objective Entrance Criteria Procedure Exit Criteria Data Output
CVE-TSC003-v01 Verify Traffic Signal Controller processes a Signal Preemption Call from the Message Handler
Message Handler is configured for "normal" operation, including receiving SRMs from an RSU and placing a priority call to a Signal Controller
Message Handler is connected to the RSU
RSU is configured for "normal" operation, including forwarding J2735 SRMs to the Message Handler
Signal Controller is configured for "normal" operation including processing Priority Calls from the Message Handler
Signal Controller is connected to the Message Handler
OBU1 is configured to broadcast authorized J2735 SRMs requesting Preemption
Power on RSU, Message Handler, OBU1, and Signal Controller
OBU1 broadcasts SRMs signed with a signal preemption certificate
RSU forwards SRM to the Message Handler
Message Handler sends a Signal Preemption Call to the Signal Controller
Signal Controller changes timing based on the preemption call
Data Recorder log the time change in the test log
Signal Controller change timing based on the preemption call
Test Log indicating Signal Controller changes timing appropriately
Chapter 8. Test Procedures
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 99
Test Case ID Objective Entrance Criteria Procedure Exit Criteria Data Output
CVETSC004-v01 Verify Traffic Signal Controller sends Priority status to the Message Handler
Message Handler is configured for "normal" operation, including receiving SRMs from an RSU and placing a priority call to a Signal Controller
Message Handler is connected to the RSU
RSU is configured for "normal" operation, including forwarding J2735 SRMs to the Message Handler
Signal Controller is configured for "normal” operation including processing Priority Calls from the Message Handler
Signal Controller is connected to the Message Handler
OBU1 is configured to broadcast authorized J2735 SRMs requesting priority
Connect a laptop (L1) running Wireshark between the Signal Controller and the Message Handler such that data can flow between the Signal Controller and the Message Handler and Wireshark can monitor the data packets.
Log received packets on L1
Power on RSU, Message Handler, OBU1, and Signal Controller
OBU1 broadcasts SRMs signed with a signal priority certificate
RSU forwards SRM to the Message Handler
Message Handler sends a Signal Priority Call to the Signal Controller
Inspect packets coming from the Signal Controller on L1 to verify the Signal Controller is sending signal priority status data to the Message Handler
Signal Controller send signal priority status to the Message Handler
pcap file from L1 containing signal status data sent from the Signal Controller to the Message Handler
Chapter 8. Test Procedures
100 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
Test Case ID Objective Entrance Criteria Procedure Exit Criteria Data Output
CVE-TSC005-v01 Verify Traffic Signal Controller sends Preemption status to the Message Handler
Message Handler is configured for "normal" operation, including receiving SRMs from an RSU and placing a preemption call to a Signal Controller
Message Handler is connected to the RSU
RSU is configured for "normal" operation, including forwarding J2735 SRMs to the Message Handler
Signal Controller is configured for "normal" operation including processing Preemption Calls from the Message Handler
Signal Controller is connected to the Message Handler
OBU1 is configured to broadcast authorized J2735 SRMs requesting preemption
Connect a laptop (L1) running Wireshark between the Signal Controller and the Message Handler such that data can flow between the Signal Controller and the Message Handler and Wireshark can monitor the data packets.
Log received packets on L1
Power on RSU, Message Handler, OBU1, and Signal Controller
OBU1 broadcasts SRMs signed with a certificate authorizing it request preemption
RSU forwards SRM to the Message Handler
Message Handler sends a Signal Preemption Call to the Signal Controller
Inspect packets coming from the Signal Controller on L1 to verify the Signal Controller is sending signal preemption status data to the Message Handler
Signal Controller send signal priority status to the Message Handler
pcap file from L1 containing signal status data sent from the Signal Controller to the Message Handler
Source: City of Columbus
Chapter 8. Test Procedures
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 101
8.6. TRAFFIC CV MANAGEMENT SYSTEM
Table 18: Traffic CV Management System Test Procedures
Test Case ID Objective Entrance Criteria Procedure Exit Criteria Data Output
CVE-TCVMS001-v01 Verify Traffic CV Management staff can input a MAP message payload to the TCVMS and specify which RSU it should be broadcast from
TCVMS is configured for "normal" operations, including sending MAP files to RSUs
TCVMS is connected to a network capable of accessing the RSU
RSU is configured for "normal" operation
RSU is connected to a network capable of accessing the TCVMS
A MAP file is stored on the TCVMS and ready to be sent to an RSU
Log into the TCVMS and configure MAP message for specific RSU
Execute TCVMS process for sending MAP messages to designated RSUs
Confirm MAP message payload matches the message sent from the TCVMS
TCVMS successfully loads MAP message to RSU
Screen shot, including date and time stamp, of RSU directory before sending MAP message
Screen shot, including date and time stamp, of RSU directory after sending MAP message, showing the payload of the MAP message
MAP message payload from TCVMS
CVE-TCVMS002-v01 Verify Traffic CV Management staff can input an RSM payload to the TCVMS and specify which RSU it should be broadcast from
TCVMS is configured for "normal" operations, including sending RSM messages to RSUs
TCVMS is connected to a network capable of accessing the RSU
RSU is configured for "normal" operation
RSU is connected to a network capable of accessing the TCVMS
An RSM message is stored on the TCVMS and ready to be sent to an RSU
Log into the TCVMS and configure RSM for specific RSU
Execute TCVMS process for sending RSM messages to designated RSUs
Confirm RSM message payload matches the message sent from the TCVMS
TCVMS successfully loads RSM to RSU
Screen shot, including date and time stamp, of RSU directory before sending RSM message
Screen shot, including date and time stamp, of RSU directory after sending RSM message, showing the payload of the RSM message
RSM message payload from TCVMS
Chapter 8. Test Procedures
102 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
Test Case ID Objective Entrance Criteria Procedure Exit Criteria Data Output
CVE-TCVMS003-v01 Verify the TCVMS archives the MAP message locally and forwards it to the Operating System
TCVMS is configured for "normal" operations, including sending MAP messages to the Operating System
TCVMS is connected to a network capable of accessing the Operating System
Operating System is configured for "normal" operation, including receiving MAP messages from the TCVMS
Operating System is connected to a network capable of accessing the TCVMS
A MAP message is stored on the TCVMS and ready to be sent to the Operating System
Log into the TCVMS and confirm MAP messages for RSU are stored locally in the designated folder
Execute TCVMS process for sending MAP messages to the Operating System
Confirm MAP message is archived in the Operating System
MAP messages are stored locally on the TCVMS
MAP messages are stored on the Operating System
Screen shot, including date and time stamp, of TCVMS directory showing MAP message payloads.
Screen shot, including date and time stamp, of Operating System directory before sending MAP message.
Screen shot, including date and time stamp, of Operating System directory after sending MAP message, showing MAP message payload.
Chapter 8. Test Procedures
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 103
Test Case ID Objective Entrance Criteria Procedure Exit Criteria Data Output
CVE-TCVMS004-v01 Verify the TCVMS archives the RSM locally and forwards it to the Operating System (Operating System)
TCVMS is configured for "normal" operations, including sending RSM messages to the Operating System
TCVMS is connected to a network capable of accessing the Operating System
Operating System is configured for "normal" operation, including receiving RSU messages from the TCVMS
Operating System is connected to a network capable of accessing the TCVMS
An RSM message is stored on the TCVMS and is ready to be sent to the Operating System
Log into the TCVMS and generate an RSM for a specific RSU
Monitor TCVMS to confirm the RSM is sent to the Operating System
Confirm the RSM just generated is archived in the Operating System
RSM messages are stored locally on the TCVMS
RSM messages are stored on the Operating System
Screen shot, including date and time stamp, of TCVMS directory showing RSM message payloads.
Screen shot, including date and time stamp, of Operating System directory before sending RSM message.
Screen shot, including date and time stamp, of Operating System directory after sending RSM message, showing RSM message payload.
Chapter 8. Test Procedures
104 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
Test Case ID Objective Entrance Criteria Procedure Exit Criteria Data Output
CVE-TCVMS005-v01 Verify the TCVMS archives BSMs locally and forwards them to the Operating System
TCVMS is configured for "normal" operations, including sending BSMs to the Operating System
TCVMS is connected to a network capable of accessing the Operating System
Operating System is configured for "normal" operation, including receiving BSMs from the TCVMS
Operating System is connected to a network capable of accessing the TCVMS
BSMs are stored on the TCVMS and ready to be sent to the Operating System
Log into the TCVMS and confirm BSMs are stored locally in the designated folder
Execute TCVMS process for sending BSMs to the Operating System
Confirm BSMs are archived in the Operating System
BSMs are stored locally on TCVMS
BSMs are stored on the Operating System
Screen shot, including date and time stamp, of TCVMS directory showing BSM payloads.
Screen shot, including date and time stamp, of Operating System directory before sending BSMs.
Screen shot, including date and time stamp, of Operating System directory after sending BSMs, showing BSM payloads.
Chapter 8. Test Procedures
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 105
Test Case ID Objective Entrance Criteria Procedure Exit Criteria Data Output
CVE-TCVMS006-v01 Verify the TCVMS archives the SPaT Message locally and forwards it to the Operating System
TCVMS is configured for "normal" operations, including sending SPaT messages to the Operating System
TCVMS is connected to a network capable of accessing the Operating System
Operating System is configured for "normal" operation, including receiving SPaT messages from the TCVMS
Operating System is connected to a network capable of accessing the TCVMS
SPaT messages are stored on the TCVMS and ready to be sent to the Operating System
Log into the TCVMS and confirm SPaT messages are stored locally in the designated folder
Execute TCVMS process for sending SPaT messages to the Operating System
Confirm SPaT messages are archived in the Operating System
SPaT messages are stored locally on TCVMS
SPaT messages are stored on the Operating System
Screen shot, including date and time stamp, of TCVMS directory showing SPaT message payloads.
Screen shot, including date and time stamp, of Operating System directory before sending SPaT messages.
Screen shot, including date and time stamp, of Operating System directory after sending SPaT messages, showing SPaT message payloads.
Chapter 8. Test Procedures
106 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
Test Case ID Objective Entrance Criteria Procedure Exit Criteria Data Output
CVE-TCVMS007-v01 Verify the TCVMS archives the SRM locally and forwards it to the Operating System
TCVMS is configured for "normal" operations, including sending SRM messages to the Operating System
TCVMS is connected to a network capable of accessing the Operating System
Operating System is configured for "normal" operation, including receiving SRM messages from the TCVMS
Operating System is connected to a network capable of accessing the TCVMS
SRM messages are stored on the TCVMS and ready to be sent to the Operating System
Log into the TCVMS and confirm SRM messages are stored locally in the designated folder
Execute TCVMS process for sending SRM messages to the Operating System
Confirm SRM messages are archived in the Operating System
SRM messages are stored locally on TCVMS
SRM messages are stored on the Operating System
Screen shot, including date and time stamp, of TCVMS directory showing SRM message payloads.
Screen shot, including date and time stamp, of Operating System directory before sending SRM messages.
Screen shot, including date and time stamp, of Operating System directory after sending SRM messages, showing SRM message payloads.
Chapter 8. Test Procedures
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 107
Test Case ID Objective Entrance Criteria Procedure Exit Criteria Data Output
CVE-TCVMS008-v01 Verify the TCVMS archives the SSM locally and forwards it to the Operating System
TCVMS is configured for "normal" operations, including sending SSM messages to the Operating System
TCVMS is connected to a network capable of accessing the Operating System
Operating System is configured for "normal" operation, including receiving SSM messages from the TCVMS
Operating System is connected to a network capable of accessing the TCVMS
SSM messages are stored on the TCVMS and ready to be sent to the Operating System
Log into the TCVMS and confirm SSM messages are stored locally in the designated folder
Execute TCVMS process for sending SSM messages to the Operating System
Confirm SSM messages are archived in the Operating System
SSM messages are stored locally on TCVMS
SSM messages are stored on the Operating System
Screen shot, including date and time stamp, of TCVMS directory showing SSM message payloads.
Screen shot, including date and time stamp, of Operating System directory before sending SSM messages.
Screen shot, including date and time stamp, of Operating System directory after sending SSM messages, showing SSM message payloads.
Chapter 8. Test Procedures
108 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
Test Case ID Objective Entrance Criteria Procedure Exit Criteria Data Output
CVE-TCVMS009-v01 Verify the TCVMS archives the RTCM locally and forwards it to the Operating System
TCVMS is configured for "normal" operations, including sending RTCM messages to the Operating System
TCVMS is connected to a network capable of accessing the Operating System
Operating System is configured for "normal" operation, including receiving RTCM messages from the TCVMS
Operating System is connected to a network capable of accessing the TCVMS
RTCM messages are stored on the TCVMS and ready to be sent to the Operating System
Log into the TCVMS and confirm RTCM messages are stored locally in the designated folder
Execute TCVMS process for sending RTCM messages to the Operating System
Confirm RTCM messages are archived in the Operating System
RTCM messages are stored locally on TCVMS
RTCM messages are stored on the Operating System
Screen shot, including date and time stamp, of TCVMS directory showing RTCM message payloads.
Screen shot, including date and time stamp, of Operating System directory before sending RTCM messages.
Screen shot, including date and time stamp, of Operating System directory after sending RTCM messages, showing RTCM message payloads.
CVE-TCVMS010-v01 Verify Traffic CV Management Staff can specify a performance metric to the TCVMS (and specify how it is calculated)
NOTE: This is a Smart Columbus Operating System Test Case
TCVMS is configured for "normal" operations, including generating performance metrics
Data related to performance metrics are stored on the TCVMS
Log into the TCVMS
Configure Performance Metric(s) to be generated in TCVMS
Execute TCVMC command to generate Performance Metric(s)
Confirm TCVMS returns request Performance Metric(s)
TCVMS returns requested Performance Metric(s)
Screen shot, including date and time stamp, of configuration of TCVMS Performance Metric(s) to be generated
Screen shot, including date and time stamp, of Performance Metric(s) returned by the TCVMS
Chapter 8. Test Procedures
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 109
Test Case ID Objective Entrance Criteria Procedure Exit Criteria Data Output
CVE-TCVMS011-v01 Verify generated performance metrics are uploaded to the Operating System.
NOTE: This is a Smart Columbus Operating System Test Case
TCVMS is configured for "normal" operations, including sending performance metrics to the Operating System. NOTE: This is a Smart Columbus Operating System Test Case
TCVMS is connected to a network capable of accessing the OS
Operating System is configured for "normal" operation, including receiving performance metrics from the TCVMS
Operating System is connected to a network capable of accessing the TCVMS
Performance metrics are stored on the TCVMS and ready to be sent to the OS
Log into the TCVMS and confirm performance metrics are stored locally in the designated folder
Execute TCVMS process for sending performance metrics to the OS
Confirm TCVMS performance metrics are archived in the OS
Performance metrics are stored locally on TCVMS
TCVMS performance metrics are stored on the OS
Screen shot, including date and time stamp, of TCVMS directory showing performance metric files.
Screen shot, including date and time stamp, of Operating System directory before sending performance metrics.
Screen shot, including date and time stamp, of Operating System directory after sending performance metrics, showing TCVMS performance metric files.
Chapter 8. Test Procedures
110 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
Test Case ID Objective Entrance Criteria Procedure Exit Criteria Data Output
CVE-TCVMS012-v01 Verify PII is removed prior to sending data to Operating System
NOTE: This is a Smart Columbus Operating System Test Case
RSU is configured for ""normal"" operation, including forwarding BSMs to the TCVMS
RSU is connected to a network capable of accessing the TCVMS
OBU is configured for "normal" operation, including broadcasting BSMs
TCVMS is configured to receive BSMs from RSUs, log "raw" (with PII) BSMs in log1, remove PII from received BSMs, log BSMs with PII removed (log2),"normal" operation, and send BSM log files to the OS
TCVMS is connected to a network capable of accessing the RSU
TCVMS is connected to a network capable of accessing the OS
Operating System is configured for "normal" operation, including receiving data from the TCVMS
Operating System is connected to a network capable of accessing the TCVMS
OBU, broadcasting BSMs, is brought into communication range of the RSU
RSU forwards BSMs to TCVMS
TCVMS receives BSMs
TCVMS logs "raw" (with PII) BSMs (log1) (for testing)
TCVMS removes PII and logs (log2) BSMs in "normal" operation
Confirm lobed BSMs in log2 do not contain PII
TCVMS removes PII from received BSMs prior to logging
BSMs without PII are stored on the OS
TCVMS log1, containing BSMs with PII
TCVMS log2, containing BSMs without PII
Operating System log files containing BSM without PII (from TCVMS log2)
Chapter 8. Test Procedures
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 111
Test Case ID Objective Entrance Criteria Procedure Exit Criteria Data Output
CVE-TCVMS013-v01 Verify Traffic CV Management Staff can specify a source for each RSU to receive RTCM data
TCVMS is configured for "normal" operations, including sending configuration parameters to RSUs
TCVMS is connected to a network capable of accessing the RSU
RSU is configured for "normal" operation, including receiving configuration parameters from the TCVMS
RSU is connected to a network capable of accessing the TCVMS
RTCM source configuration parameters are configured on the TCVMS and ready to be sent to RSUs
Log into the TCVMS and confirm RTCM source parameters
Execute TCVMS process for sending configuration parameters to RSUs
Confirm RSU is configured with the correct RTCM source parameters
RSUs are receiving RTCM messages from the specified NTRIP caster
Screen shot, including date and time stamp, of TCVMS showing RSU RTCM configuration parameters.
Screen shot, including date and time stamp, of RSU configuration before sending the RTCM source parameters.
Screen shot, including date and time stamp, of RSU configuration after sending configuration parameters, showing RTCM configuration parameters.
CVE-TCVMS014-v01 Verify TCVMS can detect unauthorized network activity.
NOTE: This is a Traffic Management Center Test Case
TCVMS is configured to send an email to operations staff if a user enters the wrong password more than 3 times
Attempt to log into the TCVMS 4 times using a wrong password
After the 4th attempt, confirm an email was sent to operations staff
TCVMS detects excessive failed user log in attempts
TCVMS sends an email to operations staff indicating excessive failed user log in attempts
Email to operations staff from the TCVMS system indicating excessive failed user log in attempts
Chapter 8. Test Procedures
112 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
Test Case ID Objective Entrance Criteria Procedure Exit Criteria Data Output
CVE-TCVMS015-v01 Verify TCVMS can detect traffic signal cabinet tampering and issue the proper notifications to Traffic CV management staff.
NOTE: This is a Traffic Management Center Test Case
Message Handler is configured for "normal" operation, including detecting Cabinet Tamper Events and sending events to the TCVMS
Message Handler is connected to a network capable of accessing the TCVMS
TCVMS is configured for "normal" operation, including receiving Cabinet Tamper Events and sending an email to operations staff
TCVMS is connected to a network capable of accessing the Message Handler
Connect a toggle switch (S1) to the Cabinet tamper detection Input/output (IO) port of the Message Handler
Set S1 to "inactive" (no event detected)
Power on the Message Handler
Confirm TCVMS is not detecting a Cabinet Tamper event
Set S1 to "active" (event detected)
Confirm TCVMS detected a Cabinet Tamper event
Upon a Cabinet Tamper event, confirm an email was sent operations staff
TCVMS detects a Cabinet Tamper event
TCVMS sends an email to operations staff indicating a Cabinet Tamper event
Email to operations staff from the TCVMS system indicating a Cabinet Tamper event
CVE-TCVMS016-v01 Verify Traffic CV Management Staff can modify configurable setting of an RSU using the TCVMS.
TCVMS is configured for "normal" operations, including sending configuration parameters to RSUs
TCVMS is connected to a network capable of accessing the RSU
RSU is configured for "normal" operation, including receiving configuration parameters from the TCVMS
RSU is connected to a network capable of accessing the TCVMS
RSU configuration parameters are configured on the TCVMS and ready to send to RSUs
Log into the TCVMS and confirm RSU configuration parameters
Execute TCVMS process for sending configuration parameters to RSUs
Confirm RSU is configured with the parameters provided by the TCVMS
RSUs are configured with parameters provided by TCVMS
Screen shot, including date and time stamp, of TCVMS showing RSU configuration parameters.
Screen shot, including date and time stamp, of RSU configuration before sending the parameters.
Screen shot, including date and time stamp, of RSU configuration after sending configuration parameters, showing RSU configuration parameters.
Chapter 8. Test Procedures
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 113
Test Case ID Objective Entrance Criteria Procedure Exit Criteria Data Output
CVE-TCVMS017-v01 Verify Traffic CV management staff have access to the following RSU data via the TCVMS: uptime status, connectivity, RSU graphical display, etc.
TCVMS is configured for "normal" operations, including pulling status elements from RSUs
TCVMS is connected to a network capable of accessing the RSU
RSU is configured for "normal" operation, including receiving requests for status elements from the TCVMS
RSU is connected to a network capable of accessing the TCVMS
Log into the TCVMS
Execute TCVMS process for retrieving status elements from RSUs
Confirm TCVMS receives status elements from RSUs
TCVMS shows status elements from RSUs
Screen shot, including date and time stamp, of RSU showing status elements.
Screen shot, including date and time stamp, of RSU status in TCVMS before requesting update.
Screen shot, including date and time stamp, of RSU status in TCVMS after requesting an update, parameters, showing RSU Status.
CVE-TCVMS018-v01 Verify alert is issued for unauthorized network activity using Dashboard icon and an email is sent to Traffic CV Management Staff
NOTE: This is a Traffic Management Center Test Case
TCVMS is configured for "normal" operations, including displaying/changing Dashboard icons and sending email alerts when unauthorized network activity is detected
TCVMS is connected to a network capable of send emails to the appropriate staff members
Log into the TCVMS and monitor the Dashboard and administrator email
From a separate console/computer, attempt to log into the TCVMS with the wrong credentials 3 times
Monitor TCVMS Dashboard for icon indicating a failed log in attempt
Monitor Administrator email for a failed log in attempt notification
TCVMS Dashboard indicates a failed login attempt
Administrator receives failed log in email notification
Screen shot, including date and time stamp, of Dashboard alert
Copy of email notification with date and time stamp
Chapter 8. Test Procedures
114 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
Test Case ID Objective Entrance Criteria Procedure Exit Criteria Data Output
CVE-TCVMS019-v01 Verify alert is issued when RSU is not operating as intended (degraded/failure conditions, or limited network connectivity) using Dashboard icon and an email is to Traffic CV Management Staff
TCVMS is configured for "normal" operations, including displaying/changing Dashboard icons and sending email alerts when an RSU is malfunctioning
TCVMS is connected to a network capable of accessing RSUs
RSU is connected to a network capable of accessing the TCVMS
Log into the TCVMS and monitor the Dashboard and administrator email
From a separate console/computer, reboot an RSU
Monitor TCVMS Dashboard for icon indicating a loss of connection to an RSU
Monitor Administrator email for a Loss of RSU Connectivity notification
TCVMS Dashboard indicates a lost RSU connection
Administrator receives Loss of RSU connection email notification
Screen shot, including date and time stamp, of Dashboard alert
Copy of email notification with date and time stamp
CVE-TCVMS020-v01 Verify TCVMS logs all alerts/emails to Traffic CV Management Staff
NOTE: This is a Traffic Management Center Test Case
TCVMS is configured for "normal" operations, including archiving alerts and email notifications
Log into the TCVMS and confirm Alerts and email notifications are stored locally in the designated folder
Alerts and email notifications are stored locally on TCVMS
Screen shot, including date and time stamp, of TCVMS directory showing stored alerts and email notifications
CVE-TCVMS021-v01 Verify Traffic CV Management Staff can query archived data on the TCVMS
NOTE: This is an Operating System Test Case
TCVMS is configured for "normal" operations, including archiving data
Log into the TCVMS and run a query for data, for a given date and time range
Confirm data matching the query criteria is returned
Returned data matches the query criteria
Screen shot, including date and time stamp, of query run against the TCVMS
Screen shot, including data and time stamp, of the data returned from the query
Source: City of Columbus
Chapter 8. Test Procedures
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 115
8.7. TRANSIT CV MANAGEMENT SYSTEM
Table 19: Transit CV Management System Test Procedures
Test Case ID Objective Entrance Criteria Procedure Exit Criteria Data Output
CVE-TrCVMS001-v01 Verify Transit CV Management Staff can specify a performance metric to the TrCVMS
TrCVMS is configured for "normal" operations, including generating performance metrics
Data related to performance metrics are stored on the TrCVMS
Log into the TrCVMS
Configure Performance Metric(s) to be generated in TrCVMS
Execute TrCVMS command to generate Performance Metric(s)
Confirm TrCVMS returns request Performance Metric(s)
TrCVMS returns requested Performance Metric(s)
Screen shot, including date and time stamp, of configuration of TrCVMS Performance Metric(s) to be generated
Screen shot, including date and time stamp, of Performance Metric(s) returned by the TrCVMS
CVE-TrCVMS002-v01 Verify generated performance metrics are uploaded to the Operating System
TrCVMS is configured for "normal" operations, including sending performance metrics to the Operating System
TrCVMS can connect to the Operating System
Operating System is configured for "normal" operation, including receiving performance metrics from the TrCVMS
Performance metrics are stored on the TrCVMS and ready to be sent to the Operating System
Log into the TrCVMS and confirm performance metrics are stored locally in the designated folder
Execute TrCVMS process for sending performance metrics to the Operating System
Confirm TrCVMS performance metrics are archived in the Operating System
Performance metrics are stored locally on TrCVMS
TrCVMS performance metrics are stored on the Operating System
Screen shot, including date and time stamp, of TrCVMS directory showing performance metric files.
Screen shot, including date and time stamp, of Operating System directory before sending performance metrics.
Screen shot, including date and time stamp, of Operating System directory after sending performance metrics, showing TrCVMS performance metric files.
Chapter 8. Test Procedures
116 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
Test Case ID Objective Entrance Criteria Procedure Exit Criteria Data Output
CVE-TrCVMS003-v01 Verify Transit CV Management Staff can query archived data on the TrCVMS
TrCVMS is configured for "normal" operations, including archiving data
Log into the TrCVMS and run a query for data, for a given date and time range
Confirm data matching the query criteria is returned
Returned data matches the query criteria
Screen shot, including date and time stamp, of query run against the TrCVMS
Screen shot, including data and time stamp, of the data returned from the query
CVE-TrCVMS004-v01 Verify the TrCVMS archives the Transit Vehicle Interaction Events and forwards them to the Operating System
TrCVMS is configured for "normal" operations, including sending Transit vehicle event logs to the Operating System
TrCVMS can connect to the Operating System
Operating System is configured for "normal" operation, including receiving Transit vehicle event logs from the TrCVMS
Transit vehicle event logs are stored on the TrCVMS and ready to be sent to the Operating System
Log into the TrCVMS and confirm Transit vehicle event logs are stored locally in the designated folder
Execute TrCVMS process for sending Transit vehicle event logs to the Operating System
Confirm Transit vehicle event logs are archived in the Operating System
Transit vehicle event logs are stored locally on TrCVMS
Transit vehicle event logs are stored on the Operating System
Screen shot, including date and time stamp, of TrCVMS directory showing Transit vehicle event logs.
Screen shot, including date and time stamp, of Operating System directory before sending Transit vehicle event logs.
Screen shot, including date and time stamp, of Operating System directory after sending Transit vehicle event logs, showing Transit vehicle event logs.
Source: City of Columbus
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 117
Appendix A. Test Result Summary
This appendix contains some of the Microsoft Excel Tools utilized to capture and document test activities.
A.1 TEST RESULTS
Table 20 shows the TRL utilized to record results of each test.
Table 20: Test Result Log
Date Name Tester Role Test ID
Test Objective
Run Number
Test Result (P/F) Comment
Source: City of Columbus
A.2 DEFECT TRACKER
The defect tracker will be used during testing to capture, track, monitor, and address anomalies observed during testing. For each entry, the development team will work to understand and reproduce (where possible) the defect, identify the root cause, summarize a response and log the activities taken to resolve the issue. As outlined in Section 4.5.4, the defect tracker helps with prioritizing defects based on severity level (critical to low) and maintains traceability to the test ID as well as status. The status field provides a simplified view of the various states a defect passes through as it moves toward resolution and closure. A defect can have the following status values:
Opened – indicates the defect has been logged and reported for correction.
Re-Opened – indicates a defect was once closed and then re-opened for modification.
Closed – indicates a defect was received, reviewed, and determined was not a defect (i.e., duplicate
entry or a request for enhancement). In these cases, no corrective action is taken, and an explanation
is provided by the development team while closing-out the defect ticket.
Canceled – indicates a scenario or test case where the defect derived was canceled and therefore
the defect is canceled by default.
Resolved – indicates a defect has been successfully reviewed, verified, and a resolution was
implemented to solve the problem along with the date the defect was corrected.
Returned – indicates the defect was returned to the tester for additional information.
Deferred – indicates the defect has been designated for correction for a later date.
In cases when a conflict arises between a design element that tied to a requirement and a software product,
the development manager will coordinate with the test manager to determine if a change to the system
design and/or requirement is appropriate. The City of Columbus project manager will carefully review all
Appendix A. Test Result Summary
118 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
requests to make a change that impacts the system design and requirements. All change requests will be
captured within the change log tool.
Table 21 shows a sample Defect Tracker for the CVE.
Table 21: Defect Tracker
Date Defect No.
Defect Description Severity
Defect Status
Test ID
Date Found
Assigned To
Resolution Description Comment
Source: City of Columbus
A.3 CHANGE REQUEST LOG
Table 22 is a sample Change Request Log for the CVE.
Table 22: Change Request Log
CR ID Description Justification Defect ID Requirement Status
Source: City of Columbus
A.4 TEST SUMMARY
Table 23 shows a sample Test Summary Report.
Table 23: Test Summary Report
Date Test Cases Planned
Test Cases Executed
Test Cases Passed
Test Cases Failed
Test Cases Deferred
Source: City of Columbus
Appendix A. Test Result Summary
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 119
Table 24 shows a sample Defect Matrix.
Table 24: Defect Matrix
Defect Release Open Closed Canceled Resolved Deferred
Source: City of Columbus
A.5 TEST CASE STATUS
Table 25 is a sample Change Request Log for the CVE.
Table 25: Test Case Status
Test ID Test Objective No. of Test Runs Expected Outcome Status
Source: City of Columbus
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 121
Appendix B. Terminology and Conventions
B.1 NUMBERING CONVENTION
Each testing element contains a unique identifier for traceability and configuration management. Test cases
and scenarios for all projects in the Smart Columbus program will follow the same convention, each
representing an identifiable attribute of the traced metric. The convention is as follows:
Figure 1: Numbering Convention
Source: City of Columbus
Table 26: Numbering Convention
Octet Description Data Type, Casing
Number of Characters or Digits
a. Project Abbreviation The designated Smart Columbus project acronym (i.e., CVE)
String, upper case
Variable
b. Test Type Code • XYZ: Test Case for the XYZ
• XYZ: Test Case for the XYZ
• XYZ: Test Case for the XYZ
• XYZ: Test Case for the XYZ
• TES: Test Scenario
String, upper case
2
c. Test Number Integer incrementing by 1, indicating the number of requirements established
Integer 3
d. “V” Static Character Static letter “V” represents the version for the particular test objective and procedure
Character 1
e. Version Number Integer incrementing by 1, indicating the number of revisions made to the test element being traced
Integer 2
Source: City of Columbus
Project Abbreviation
01
Test Type Code
Test Number V Static Character
Version Number
02 03
(a) (b) (c) (d) (e)
1.
Appendix B. Terminology and Conventions
122 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
An example of a test case for the integration of CVE would be CVE-DSRC001-V01.
1. “CVE” is the project abbreviation.
2. “DSRC001” is the test type code coupled with the three-digit test number.
3. “V01” is the static “V” coupled with the two-digit version number.
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 123
Appendix C. Acronyms and Abbreviations
Table 27 contains project specific acronyms used throughout this document.
Table 27: Acronym List
Abbreviation/Acronym Definition
ARC-IT Architecture Reference for Cooperative and Intelligent Transportation
BSM Basic Safety Message
COTA Central Ohio Transit Authority
CTSS Columbus Traffic Signal System
CV Connected Vehicle
CVE Connected Vehicle Environment
CVRIA Connected Vehicle Reference Implementation Architecture
CVTT Connected Vehicle Test Tool
EEBL Emergency Electronic Brake Light
EV Emergency Vehicle
FCW Forward Collision Warning
HDV Heavy-Duty Vehicle
IMA Intersection Movement Assist
IT Information Technology
ITS Intelligent Transportation Systems
LCW Lane Change Warning
LDV Light-Duty Vehicle
MAP Map
MTP Master Test Plan
OBU Onboard Unit
Operating System Smart Columbus Operating System
OSADP Open Source Application Development Portal
RLVW Red Light Violation Warning
RSSZ Reduced Speed School Zone
RSU Roadside Unit
RTCM Radio Technical Commission for Maritime Services
SCMS State of Ohio Security Credential Management System
Appendix C. Acronyms and Abbreviations
124 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Draft Report
Abbreviation/Acronym Definition
SPaT Signal Phase and Timing
SRM Signal Request Message
SSM Signal Status Message
TCVMS Traffic Connected Vehicle Management System
TMC Traffic Management Center
TrCVMS Transit Connected Vehicle Management System
TSP Transit Signal Priority
USDOT U.S. Department of Transportation
V2I Vehicle-to-Infrastructure
V2V Vehicle-to-Vehicle
Source: City of Columbus
Connected Vehicle Environment (CVE) Test Plan – Draft Report | Smart Columbus Program | 125
Appendix D. Glossary
Table 28 contains definitions of project-specific terms used throughout this document.
Table 28: Glossary
Term Definition
CV Test Tool (CVTT) The CVTT is a specialized DSRC Onboard Unit designed to capture DSRC messages for decoding, visualizing DSRC message (e.g. MAP, RSM), and running CV applications for system acceptance
Position Correction System
A Position Correction System provides Global Navigation Satellite System (GNSS) data consisting of carrier phase and code range measurements in support of three dimensional positioning. Correction Data is used to improve the precision of GPS data. Enhanced post-processed coordinates approach a few centimeters relative to the National Spatial Reference System, both horizontally and vertically.
Smart Columbus Operating System The Operating System collects and archives CV data for distribution to three parties.
Traffic CV Management Center (TCVMS)
The TCVMS is the operation and maintenance enter of the CVE. RSU are monitored from, and data is archived to, the TCVMS
Transit CV Management Center (TrCVMS)
The TrCVMS is the CV data center for COTA. Transit OBUs upload the log files to the TrCVMS and TrCVMS, and staff can review and process transit vehicle log files
Source: City of Columbus