Evaluation of Travel Information and Prediction System (TIPS) for Work Zone Operations
University of Maryland, College Park
2
TIPS overview
Traffic Information and Prediction System
• The messages displayed on these three PCMS are the range of travel time from each spot to Spot 5 (I-695 Gore) (e.g., “12 TO 16 MIN TO I-695”).
Spot 1
Spot 2 Spot 3
Spot 4 Spot 5
4.55 miles
7.55 miles
9.75 miles
10.80 miles
PCMS 1
PCMS 2 PCMS 3
SensorTrailer
I-695Gore
3
Criteria of TIPS Evaluation
System Accuracy and Reliability
Data collection for evaluation: Testing vehicle method
• TIPS Accuracy : comparing the actual measured travel times to those displayed from each PCMS.
• TIPS Reliability : consistency between the travel time information reported on the website and those displayed on each PCMS.
4
System Accuracy Evaluation on 11/14
System Accuracy
(Note: “No message” data on PCMS #1 were excluded from the sample size)
PCMS #1 PCMS #2 PCMS #3
LB UB LB UB LB UB
1750 vph
2400 vph
2200 vph
4250 vph
1650 vph
2650 vph
# of incorrect 9* 8 7
# of correct 6* 27 27System accuracy
(%) 40.00 77.14 79.41
• The accuracy drops as PCMS #1 is far away from I-695.
Volume difference from Sensor data
5
System Accuracy Evaluation on 11/14
System Accuracy
• Statistical analysis of predicted travel time errors PCMS #1 PCMS #2 PCMS #3
Average (seconds) 149.11 38.75 206.29
Standard deviation 415.05 308.11 181.70
Deviation of predicted travel time error (sec)
Frequency
PCMS #1 PCMS #2 PCMS #3
<= -480 0 1 0
<= -240 2 0 0
<= 0 2 2 1
<= 240 1 2 3
<= 480 1 3 3
> 480 3 0 0
Total 9 8 7
• Frequency distribution of predicted travel time errors
6
System Accuracy Evaluation on 11/19
System Accuracy
11
PCMS #1 PCMS #2 PCMS #3
LB UB LB UB LB UB
1900 vph
5000 vph
2100 vph
4100 vph
1100 vph
2400 vph
# of incorrect 8 5 4
# of correct 14 15System accuracy
(%) 57.89 73.68 78.95
• The volume difference from the sensor data at PCMS #1 is greater than those at PCMS #2 and #3
• The accuracy drops as PCMS #1 is far away from I-695 and the volume difference increases.
Volume difference from Sensor data
7
System Accuracy Evaluation on 11/19
System Accuracy
PCMS #1 PCMS #2 PCMS #3
Average (seconds) -21.00 85.40 87.25
Standard deviation 315.37 268.95 401.38
Deviation of predicted travel time error (sec)
Frequency
PCMS #1 PCMS #2 PCMS #3
<= -480 0 0 1
<= -240 1 0 0
<= 0 4 2 0
<= 240 1 2 2
<= 480 1 1 1
> 480 1 0 0
Total 8 5 4
• Statistical analysis of predicted travel time errors
• Frequency distribution of predicted travel time errors
8
System Accuracy Evaluation on 11/24
System Accuracy
• The LB and UB of volumes are low and their differences are small, which mean that traffic conditions are uncongested and stable.
• The system can achieve a high accuracy under the stable traffic flow patterns.
21
PCMS #1 PCMS #2 PCMS #3
LB UB LB UB LB UB
1550 vph
2450 vph
1200vph
2400vph
1000vph
2300vph
# of incorrect 0 0 2
# of correct 21 19
System accuracy (%)
100.00 100.00 90.48
Volume difference from Sensor data
9
System Accuracy Evaluation on 11/24
System Accuracy
PCMS #1 PCMS #2 PCMS #3Average (seconds) N/A N/A 148.00Standard deviation N/A N/A 16.97
• Statistical analysis of predicted travel time errors
• Frequency distribution of predicted travel time errorsDeviation of
predicted travel time error (sec)
Frequency
PCMS #1 PCMS #2 PCMS #3
<= -480 0 0 0
<= -240 0 0 0
<= 0 0 0 0
<= 240 0 0 2
<= 480 0 0 0
> 480 0 0 0
Total 0 0 2
10
System Accuracy Evaluation on 11/25
System Accuracy
2
PCMS #1 PCMS #2 PCMS #3
LB UB LB UB LB UB
600 vph
3750 vph
1000 vph
4300 vph
1500 vph
2750 vph
# of incorrect 29 12 6
# of correct 19 25System accuracy
(%) 6.45 61.29 80.65
Volume difference from Sensor data
• The performance of such a system clearly varies with the range of flow rate variation.
• PCMS #1 experiences the largest range of flow variation, and yields the poorest results.
11
System Accuracy Evaluation on 11/25
System Accuracy
PCMS #1 PCMS #2 PCMS #3
Average (seconds) -185.52 103.92 157.50
Standard deviation 792.84 224.49 168.95
• Statistical analysis of predicted travel time errors
• Frequency distribution of predicted travel time errorsDeviation of
predicted travel time error (sec)
Frequency
PCMS #1 PCMS #2 PCMS #3
<= -480 12 0 0
<= -240 4 0 0
<= 0 1 4 1
<= 240 1 5 4
<= 480 1 2 1
> 480 10 1 0
Total 29 12 6
12
System Accuracy Evaluation on 12/04
System Accuracy
• The accuracy drops as PCMS #1 is far away from I-695 and the volume difference increases.
10
PCMS #1 PCMS #2 PCMS #3
LB UB LBUB LB UB
1200 vph
3000 vph
1100 vph
4400 vph
1500 vph
2700 vph
# of incorrect 3 0 1
# of correct 13 12System accuracy
(%) 76.92 100.00 92.31
UB
4400 vph
Volume difference from Sensor data
13
System Accuracy Evaluation on 12/04
System Accuracy
PCMS #1 PCMS #2 PCMS #3
Average (seconds) -70.00 N/A 151.00
Standard deviation 190.13 N/A N/A
• Statistical analysis of predicted travel time errors
• Frequency distribution of predicted travel time errorsDeviation of
predicted travel time error (sec)
Frequency
PCMS #1 PCMS #2 PCMS #3
<= -480 0 0 0
<= -240 0 0 0
<= 0 1 0 0
<= 240 1 0 1
<= 480 0 0 0
> 480 0 0 0
Total 2 0 1
14
System Accuracy Evaluation
Conclusions
• The accuracy is often dropped for those PCMS far away from Spot 5 (I-695 Gore), or experiencing a wide range of the flow rate variation.
• The system doesn’t provide reliable travel time information during congested peak hours, especially for a short peak or a transition period between off-peak and peak hours.
15
System Reliability Evaluation on 11/14 and 11/19
• 11/14
• 11/19
PCMS #1 PCMS #2 PCMS #3
# of Inconsistence 0 0 1
# of Consistence 19 19 18System reliability
(%)100.00 100.00 94.74
PCMS #1 PCMS #2 PCMS #3
# of Inconsistence 4 1 1
# of Consistence 9 33 30System reliability
(%)69.23 97.06 96.77
(Note: “No message” from the PCMS is excluded from the sample size.)
16
System Reliability Evaluation on 11/24 and 11/25
PCMS #1 PCMS #2 PCMS #3
# of Inconsistence 0 0 0
# of Consistence 21 21 21System reliability
(%)100.00 100.00 100.00
• 11/24
• 11/25
PCMS #1 PCMS #2 PCMS #3
# of Inconsistence 24 1 1
# of Consistence 9 33 30System reliability
(%)27.27 97.06 96.77
(Note: “Blank” from the website is included from the sample size.)
17
System Reliability Evaluation on 12/04
• 12/04
PCMS #1 PCMS #2 PCMS #3
# of Inconsistence 7 0 0
# of Consistence 6 13 13System reliability
(%)46.15 100.00 100.00
(Note: “Blank” from the website is included from the sample size.)
18
System Reliability Evaluation
Conclusions
• The system reliability also show patterns similar to the system accuracy.
• That is, the reliability of a PCMS decreases with its distance to Spot 5 (I-695 Gore).