WHITE PAPER

©2013 Mellanox Technologies. All rights reserved.

Highly Accurate Time Synchronization with ConnectX®-3 and TimeKeeper®

Highlights and Benefits:

• Sub-microsecond time synchronization, using off-the-shelf hardware

• Support for both PTP and NTP protocols and master clocks

• TimeKeeper® is easily deployable on any existing ConnectX®-3 network adapter card

Upgrading your trading platforms to reliable and precise time is achievable at a low cost and a rapid deployment model via the combination of Mellanox’s ConnectX-3 network adapter cards and TimeKeeper Client software. TimeKeeper can assure sub-microsecond time precision from both the newer IEEE 1588 Precision Time Protocol (PTP) or the standard Network Time Protocol (NTP) over shared (not dedicated) network links. Flexibility in time sources and automatic adapability to network quality allows for incremental changes to enterprise systems, and immediate high precision timing in critical components while less critical components see incremental performance improvement. For high quality links and time feeds, applications can see time locked to reference well within 500 nanoseconds of variation.

-1

-0.5

0

0.5

1

Precision in microseconds/NTPmedium quality link

Mic

roS

econ

ds

Samples

Figure 1. Precision in Microseconds/NTP Medium Quality Link

High performance comes from the complementary action of two technologies: TimeKeeper’s state-of-the-art time synchronization and management and the Mellanox ConnectX-3 hardware based packet time- stamping. TimeKeeper uses information that the ConnectX-3 provides to factor out packet delay variation (PDV) caused by buffering, network stacks, and OS scheduling and automatically improves precision of locking the local timer to the reference time. Because the ConnectX-3 can time-stamp both PTP and NTP packets, TimeKeeper can greatly improve performance of both protocols.

Figure 2. Precision in Microseconds/PTP Medium Quality Link

March 2013

In collaboration with

FSMLabs

3

-1-0.5

00.5

1

Precision in microseconds/PTP medium quality link

Samples

Mic

roS

econ

ds

page 2WHITE PAPER: Highly Accurate Time Synchronization with ConnectX®-3 and TimeKeeper®

©2013 Mellanox Technologies. All rights reserved.

TimeKeeper’s Client software will automatically detect and take advantage of the time stamp capabilities of the ConnectX-3 – the only configuration needed, is to point the software at the right network interface, IEEE 1588 and with NTP. After which, TimeKeeper can also be used to setup failover, auditable trace logs, automatic spoof and equipment failure detection, and automatic notification of error conditions to management consoles (via SMTP) and IT staff (via email). Customers can also use the web interface to validate network setup and to optimize their time distribution networks.

The graph below, Figure 3, is from the TimeKeeper web management console and shows two time sources being tracked and validated against the other. Discrepancies can trigger a switch of time sources or an alert or both- depending on configuration. TimeKeeper can track any number of independent time sources. This increases system resilience and also produces a trace of system behavior to pinpoint causes of errors. The free software for PTP and NTP can fail silently – allowing application code to keep trading on the basis of incorrect time. The “time map”, Figure 4, capability introduced in TimeKeeper 6 discovers the surrounding time distribution network and helps IT staff find configuration problems and identify sources of network time fragility. This feature is in use in some of the world’s leading financial trading organizations and has uncovered numerous previously undetected system vulnerabilities

Figure 3. Cross Check Second

Figure 4. Time Map

Because TimeKeeper and the ConnectX-3 are protocol agnostic, it is not necessary to make huge, disruptive changes in existing time distribution networks. Various parts of the network can use PTP or NTP as needed, leveraging the strengths of each and the fault tolerance derived from using sources carried over both protocols.

2266689500

Sour

ce ti

ck ra

te (p

er s

econ

d) 2266689000

2266688500

2266688000

2266687500

2266687000

2266686500

2266686000

2266685500

Samples

Source 0 ideal tickrate

Source 1 ideal tickrate

132.163.4.103

ACTS

this_system

64.113.32.5128.252.19.1:GPS

128.252.19.1

128.7.1.66129.250.35.250

204.42.158.152 129.7.1.66:GPS

129.250.35.251

143.166.82.251

166.255.32199.4.29.166

192.5.41.40

128.4.1.1

128.4.1.1. PPS

18.26.4.105.PPS

209.81.9.7:GPS

209.81.9.7

128.9.176.30:GPS

128.9.176.30

128.67.113.92

108.71.253.20:GPS

216.218.254.202:CDMA

216.218.254.202

216.45.57.38

216.171.124.36

43.77.130.254

108.71.253.20

204.123.2.5:GPS204.123.2.5

69.36.224.15

132.163.4.101

172.16.1.36

69.36.224.15:GPS

69.25.96.13

129.6.15.28

128.138.140.44

128.9.57.133

18.26.4.105

192.5.41.40:IRIG

209.51.161.238:CDMA

209.51.161.238

132.163.4.103

ACTS

this_system

64.113.32.5128.252.19.1:GPS

128.252.19.1

128.7.1.66129.250.35.250

204.42.158.152 129.7.1.66:GPS

129.250.35.251

143.166.82.251

166.255.32199.4.29.166

192.5.41.40

128.4.1.1

128.4.1.1. PPS

18.26.4.105.PPS

209.81.9.7:GPS

209.81.9.7

128.9.176.30:GPS

128.9.176.30

128.67.113.92

108.71.253.20:GPS

216.218.254.202:CDMA

216.218.254.202

216.45.57.38

216.171.124.36

43.77.130.254

108.71.253.20

204.123.2.5:GPS204.123.2.5

69.36.224.15

132.163.4.101

172.16.1.36

69.36.224.15:GPS

69.25.96.13

129.6.15.28

128.138.140.44

128.9.57.133

18.26.4.105

192.5.41.40:IRIG

209.51.161.238:CDMA

209.51.161.238

page 3WHITE PAPER: Highly Accurate Time Synchronization with ConnectX®-3 and TimeKeeper®

350 Oakmead Parkway, Suite 100, Sunnyvale, CA 94085Tel: 408-970-3400 • Fax: 408-970-3403www.mellanox.com

© Copyright 2013. Mellanox Technologies. All rights reserved.Mellanox, BridgeX, ConnectX, CORE-Direct, InfiniBridge, InfiniHost, InfiniScale, MLNX-OS, PhyX, SwitchX, UFM, Virtual Protocol Interconnect and Voltaire are registered trademarks of Mellanox Technologies, Ltd. Connect-IB, CoolBox, FabricIT, Mellanox Federal Systems, Mellanox Software Defined Storage, MetroX, Open Ethernet, ScalableHPC and Unbreakable-Link are trademarks of Mellanox Technologies, Ltd. All other trademarks are property of their respective owners. 15-513WP Rev 1.0

Figure 5. Distribution of NTP/PTP in Upgraded TimeKeeper Network

About Mellanox ConnectX-3 Network Adapters:

Mellanox’s cost-effective, high-performance, ConnectX-3 network adapters support a variety of network protocols from 1/10/40GbE and FDR 56Gb/s InfiniBand. ConnectX-3 network adapters are available through OEM server vendors in a variety of form factors, including: LOM, add-on cards, blade mezzanine cards and PCI expansion cards. With ConnectX-3 network adapters, enterprise server engineers can utilize a single, cost-effective adapter for all of their data center needs – trading, time synchronization, database, and virtualization.

For more information on Mellanox ConnectX-3 network adapter card, visit: http://www.mellanox.com/page/products_overview

For more information on TimeKeeper, contact: sales@fsmlabs.com

TimeKeeper and FSMLabs are registered trademarks of Finite State Machine Labs Inc.

TK enabledApplication Server

Application Server

Distribution of NTP/PTP in Upgraded TK Network 2

Switch

TK GrandMasterAppliance

TK enabledApplication Server

TK enabledApplication Server

Application Server

Switch

Application StratumServer

NTP PTP

NTPPTP

Application Server