Introduction
• A look at some issues regarding performance benchmarks in EMME/2
• Show a sample of benchmarks achieved by a range of hardware on the same EMME/2 model
• Hopefully stimulate some debate!
Purpose (1)
Inspired by discussions on the INRO Lists regarding the influence of the following features on performance:
• Operating System (Windows, Unix, Linux)
• Speed of Processors
• Number of Processors
• Amount of memory
• Disk type (SCSI, RAID Arrays)
Purpose (2)
Winnipeg Model:
• The Winnipeg model runs ‘too quickly’ to get meaningful benchmarks on current hardware
• Some users have recently wondered why there are not more complex elements in the Winnipeg model, such as more complex turn penalties.
Meanwhile, Some Benchmarks
PLANET South AM Benchmarks1381 zones, 1855 lines, 38764 segments, 24 MSA Iterations, 3 Trip Purposes
100
52
65
4034
26
0
20
40
60
80
100
120
PIII 733MHz128Mb
133FSBWin2k
Athlon XP1.33GHz512Mb
266FSB XP
PIV 1.8GHz512Mb
400FSBWin2k
PIV 2.6GHz512Mb
800FSBWin2k
Dual Xeon3.06GHz
2Gb533FSB XP
SCSI
Dual Xeon3.2GHz HT
2Gb800FSB XP
SCSIRAID0
Age of Hardware
Ru
n T
ime
Minutes
Some Observations (1)
• As expected, run times decrease as spec increases
• AMD Athlons look promising in terms of performance per clock cycle
• Front Side Bus (FSB) appears to be a significant indicator of bottleneck
• Athlon 64s should be investigated due to no FSB bottleneck and performance per clock cycle
Some Observations (2)
• Generally, ‘more is more’, however:
• Hyperthreaded or Multiple Processors do not speed up run times, as expected
BUT:
• In the real world, the potential to do other things at the same time as a model run IS vastly increased by more physical or virtual processors
Some Observations (3)
Rule of Thumb (Assignment) - Zvi Leve, INRO:
• Highway assignment is more influenced by processor speed
• Transit assignment is more influenced by disk speed
Some Observations (4)
Rule of Thumb (Operating Systems) – Mike Florian, INRO:
• Linux: faster for disk access for matrix calculations
• Windows: faster for assignment
A Larger Demonstration Databank?
• A larger model available to all users would provide a large sample of benchmarks
• There would be an opportunity to further showcase the potential of EMME/2 and ENIF, such as:– More complex turn penalties
– Crowding on Transit Services
– Park & Ride– Combined Assignment-Distribution-Mode Choice
techniques such as the Santiago Model (Michael Florian and Shuguang He, 11th EEUG, Madrid, 2002)
An Alternative Idea
• A ‘fictitious’ large model could be constructed:
• Network Data– Constructed from publicly available GIS data
(freely available in the US)
• Matrix Data– Purely synthetic data, perhaps constructed using
a gravity model
Conclusion
• Performance benchmarks could be more easily obtained on a more complex or larger model – Difficult to form conclusions based on a model not publicly available
• This could be achieved by:– More complexity for the Winnipeg model
– A different, larger model – real or fictitious!
• This could have the by-product of showcasing more EMME/2 and ENIF features